{"action":"closed","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/17356","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/17356/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/17356/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/17356/events","html_url":"https://github.com/elastic/elasticsearch/issues/17356","id":143687412,"number":17356,"title":"ElasticSearch and kibana timeout problem","user":{"login":"Mobasher-NetLinks","id":15998215,"avatar_url":"https://avatars.githubusercontent.com/u/15998215?v=3","gravatar_id":"","url":"https://api.github.com/users/Mobasher-NetLinks","html_url":"https://github.com/Mobasher-NetLinks","followers_url":"https://api.github.com/users/Mobasher-NetLinks/followers","following_url":"https://api.github.com/users/Mobasher-NetLinks/following{/other_user}","gists_url":"https://api.github.com/users/Mobasher-NetLinks/gists{/gist_id}","starred_url":"https://api.github.com/users/Mobasher-NetLinks/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Mobasher-NetLinks/subscriptions","organizations_url":"https://api.github.com/users/Mobasher-NetLinks/orgs","repos_url":"https://api.github.com/users/Mobasher-NetLinks/repos","events_url":"https://api.github.com/users/Mobasher-NetLinks/events{/privacy}","received_events_url":"https://api.github.com/users/Mobasher-NetLinks/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-03-26T10:40:31Z","updated_at":"2016-03-30T18:18:55Z","closed_at":"2016-03-30T18:18:55Z","body":"I'm using ElasticSearch, everything was normal and working, but message `high desk watermark [90%].....some shards will be relocated away from this node` was showed.\r\nBut after some days kibana and elasticsearch can't connect, and connection timeout occur.\r\n\r\n    Ahmads-MacBook-Pro:elasticsearch-1.7.0 mobasherfasihy$ bin/elasticsearch\r\n    [2016-03-26 14:57:57,230][INFO ][node                     ] [Sage] version[1.7.0], pid[86359], build[929b973/2015-07-16T14:31:07Z]\r\n    [2016-03-26 14:57:57,231][INFO ][node                     ] [Sage] initializing ...\r\n    [2016-03-26 14:57:57,391][INFO ][com.floragunn.searchguard.SearchGuardPlugin] Class enhancements for DLS/FLS successful\r\n    [2016-03-26 14:57:57,407][INFO ][plugins                  ] [Sage] loaded [searchguard, license, marvel], sites [marvel]\r\n    [2016-03-26 14:57:57,437][INFO ][env                      ] [Sage] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [17.9gb], net total_space [369.5gb], types [hfs]\r\n    [2016-03-26 14:57:59,083][WARN ][com.floragunn.searchguard.util.SecurityUtil] AES 256 not supported, max key length for AES is 128. To enable AES 256 install 'Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files'\r\n    [2016-03-26 14:57:59,261][INFO ][node                     ] [Sage] initialized\r\n    [2016-03-26 14:57:59,261][INFO ][node                     ] [Sage] starting ...\r\n    [2016-03-26 14:57:59,333][INFO ][transport                ] [Sage] bound_address {inet[/127.0.0.1:9304]}, publish_address {inet[localhost/127.0.0.1:9304]}\r\n    [2016-03-26 14:57:59,345][INFO ][discovery                ] [Sage] elasticsearch/XR6DsFmbQk267g3UFpbQCw\r\n    [2016-03-26 14:58:03,125][INFO ][cluster.service          ] [Sage] new_master [Sage][XR6DsFmbQk267g3UFpbQCw][Ahmads-MacBook-Pro.local][inet[localhost/127.0.0.1:9304]], reason: zen-disco-join (elected_as_master)\r\n    [2016-03-26 14:58:03,149][INFO ][http                     ] [Sage] bound_address {inet[/127.0.0.1:9204]}, publish_address {inet[localhost/127.0.0.1:9204]}\r\n    [2016-03-26 14:58:03,149][INFO ][node                     ] [Sage] started\r\n    [2016-03-26 14:58:03,157][INFO ][gateway                  ] [Sage] recovered [0] indices into cluster_state\r\n    [2016-03-26 14:58:15,288][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:58:15,288][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:58:21,293][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:58:21,294][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:58:27,304][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:58:27,304][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:58:33,136][WARN ][cluster.routing.allocation.decider] [Sage] high disk watermark [90%] exceeded on [XR6DsFmbQk267g3UFpbQCw][Sage] free: 17.9gb[4.8%], shards will be relocated away from this node\r\n    [2016-03-26 14:58:33,136][INFO ][cluster.routing.allocation.decider] [Sage] high disk watermark exceeded on one or more nodes, rerouting shards\r\n    [2016-03-26 14:58:33,315][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:58:33,315][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:58:49,326][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:58:49,326][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:58:55,328][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:58:55,328][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:59:01,330][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:59:01,330][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:59:03,133][WARN ][cluster.routing.allocation.decider] [Sage] high disk watermark [90%] exceeded on [XR6DsFmbQk267g3UFpbQCw][Sage] free: 17.9gb[4.8%], shards will be relocated away from this node\r\n    [2016-03-26 14:59:07,338][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:59:07,338][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:59:23,346][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:59:23,346][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:59:29,351][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:59:29,351][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n    [2016-03-26 14:59:33,136][WARN ][cluster.routing.allocation.decider] [Sage] high disk watermark [90%] exceeded on [XR6DsFmbQk267g3UFpbQCw][Sage] free: 17.9gb[4.8%], shards will be relocated away from this node\r\n    [2016-03-26 14:59:33,136][INFO ][cluster.routing.allocation.decider] [Sage] high disk watermark exceeded on one or more nodes, rerouting shards\r\n    [2016-03-26 14:59:35,355][ERROR][marvel.agent.exporter    ] [Sage] error connecting to [127.0.0.1:9200] [connect timed out]\r\n    [2016-03-26 14:59:35,355][ERROR][marvel.agent.exporter    ] [Sage] could not connect to any configured elasticsearch instances: [127.0.0.1:9200]\r\n\r\nand in kibana \r\n\r\n    {\"name\":\"Kibana\",\"hostname\":\"Ahmads-MacBook-Pro.local\",\"pid\":85748,\"level\":50,\"err\":{\"message\":\"Request Timeout after 5000ms\",\"name\":\"Error\",\"stack\":\"Error: Request Timeout after 5000ms\\n    at null.<anonymous> (/Users/ahmadhusseinrezae/kibana-4.1.3-darwin-x64/src/node_modules/elasticsearch/src/lib/transport.js:282:15)\\n    at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)\"},\"msg\":\"\",\"time\":\"2016-03-26T08:55:35.517Z\",\"v\":0}"}}