{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/18294","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18294/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18294/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18294/events","html_url":"https://github.com/elastic/elasticsearch/issues/18294","id":154523629,"number":18294,"title":"Cluster will not automatically assign shards","user":{"login":"alicegoldfuss","id":4809374,"avatar_url":"https://avatars.githubusercontent.com/u/4809374?v=3","gravatar_id":"","url":"https://api.github.com/users/alicegoldfuss","html_url":"https://github.com/alicegoldfuss","followers_url":"https://api.github.com/users/alicegoldfuss/followers","following_url":"https://api.github.com/users/alicegoldfuss/following{/other_user}","gists_url":"https://api.github.com/users/alicegoldfuss/gists{/gist_id}","starred_url":"https://api.github.com/users/alicegoldfuss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alicegoldfuss/subscriptions","organizations_url":"https://api.github.com/users/alicegoldfuss/orgs","repos_url":"https://api.github.com/users/alicegoldfuss/repos","events_url":"https://api.github.com/users/alicegoldfuss/events{/privacy}","received_events_url":"https://api.github.com/users/alicegoldfuss/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Allocation","name":":Allocation","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/feedback_needed","name":"feedback_needed","color":"d4c5f9"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-05-12T16:18:02Z","updated_at":"2016-05-12T17:41:27Z","closed_at":null,"body":"<!--\r\nGitHub is reserved for bug reports and feature requests. The best place\r\nto ask a general question is at the Elastic Discourse forums at\r\nhttps://discuss.elastic.co. If you are in fact posting a bug report or\r\na feature request, please include one and only one of the below blocks\r\nin your new issue.\r\n-->\r\n\r\n<!--\r\nIf you are filing a bug report, please remove the below feature\r\nrequest block and provide responses for all of the below items.\r\n-->\r\n\r\n**Elasticsearch version**: 2.2.1\r\n\r\n**JVM version**:  java version \"1.8.0_31\"\r\nJava(TM) SE Runtime Environment (build 1.8.0_31-b13)\r\nJava HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)\r\n\r\n**OS version**: CentOS release 6.7 (Final)\r\n\r\n**Description of the problem including expected versus actual behavior**:\r\n\r\nThanks in advance for your help.\r\n\r\nI inherited a poorly-behaving Elasticsearch cluster that I've been tuning on and off for a few months. Some things have improved, others have not.\r\n\r\nAbout a month ago, another team needed to reboot the (AWS hosted) nodes. I turned off shard allocation in the cluster with an API call and got a success response. However, it didn't take. So I also turned off allocation in kopf. But, rebooting a node caused shards to be allocated anyway. So we rebooted one box per day and let the cluster rebalance each time.\r\n\r\nSome time after the reboots, I noticed the cluster was still yellow. It had stopped allocating shards. I figured the API call had finally taken, so I turned allocation back on (and got another success message). According to kopf, allocation is enabled. But the cluster did not automatically assign shards.\r\n\r\nI tried toggling shard allocation on and off with both kopf and the API. But the cluster would not allocate shards.\r\n\r\nI complained about this on Twitter and an Elastic employee recommended upgrading. So, I upgraded the cluster from 1.5.2 to 2.2.1. However, the behavior persists.\r\n\r\nIf I assign shards manually with the API, they allocate just fine.\r\n\r\n**Steps to reproduce**:\r\n 1. Turn off shard allocation with API call and kopf\r\n 2. Turn on shard allocation with API call and kopf\r\n 3. Cry a lot\r\n\r\n**Provide logs (if relevant)**: I didn't see any relevant logs, but if you tell me what to look for I can grep. I'm happy to gather diagnostics for you, but please note that I no longer own this cluster as of 3pm PDT Friday.\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/218831464","html_url":"https://github.com/elastic/elasticsearch/issues/18294#issuecomment-218831464","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18294","id":218831464,"user":{"login":"alicegoldfuss","id":4809374,"avatar_url":"https://avatars.githubusercontent.com/u/4809374?v=3","gravatar_id":"","url":"https://api.github.com/users/alicegoldfuss","html_url":"https://github.com/alicegoldfuss","followers_url":"https://api.github.com/users/alicegoldfuss/followers","following_url":"https://api.github.com/users/alicegoldfuss/following{/other_user}","gists_url":"https://api.github.com/users/alicegoldfuss/gists{/gist_id}","starred_url":"https://api.github.com/users/alicegoldfuss/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alicegoldfuss/subscriptions","organizations_url":"https://api.github.com/users/alicegoldfuss/orgs","repos_url":"https://api.github.com/users/alicegoldfuss/repos","events_url":"https://api.github.com/users/alicegoldfuss/events{/privacy}","received_events_url":"https://api.github.com/users/alicegoldfuss/received_events","type":"User","site_admin":false},"created_at":"2016-05-12T17:41:27Z","updated_at":"2016-05-12T17:41:27Z","body":"Hi Clinton!\r\n\r\n```\r\n$ curl -XPUT http://node-1.com:9200/_cluster/settings -d '{\"transient\":{\"cluster.routing.allocation.enable\": \"all\"}}'\r\n{\"acknowledged\":true,\"persistent\":{},\"transient\":{\"cluster\":{\"routing\":{\"allocation\":{\"enable\":\"all\"}}}}}\r\n```"}}