{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/28623","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/28623/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/28623/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/28623/events","html_url":"https://github.com/elastic/elasticsearch/pull/28623","id":296174894,"number":28623,"title":"when start recover,check the tomstones and indices to aviod deleted index rejoining cluster","user":{"login":"huafeisuperman","id":35583861,"avatar_url":"https://avatars3.githubusercontent.com/u/35583861?v=4","gravatar_id":"","url":"https://api.github.com/users/huafeisuperman","html_url":"https://github.com/huafeisuperman","followers_url":"https://api.github.com/users/huafeisuperman/followers","following_url":"https://api.github.com/users/huafeisuperman/following{/other_user}","gists_url":"https://api.github.com/users/huafeisuperman/gists{/gist_id}","starred_url":"https://api.github.com/users/huafeisuperman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huafeisuperman/subscriptions","organizations_url":"https://api.github.com/users/huafeisuperman/orgs","repos_url":"https://api.github.com/users/huafeisuperman/repos","events_url":"https://api.github.com/users/huafeisuperman/events{/privacy}","received_events_url":"https://api.github.com/users/huafeisuperman/received_events","type":"User","site_admin":false},"labels":[{"id":836504707,"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Distributed/Distributed","name":":Distributed/Distributed","color":"0e8a16","default":false},{"id":23173,"url":"https://api.github.com/repos/elastic/elasticsearch/labels/%3Ebug","name":">bug","color":"b60205","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2018-02-11T09:17:16Z","updated_at":"2018-03-08T16:38:21Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/elastic/elasticsearch/pulls/28623","html_url":"https://github.com/elastic/elasticsearch/pull/28623","diff_url":"https://github.com/elastic/elasticsearch/pull/28623.diff","patch_url":"https://github.com/elastic/elasticsearch/pull/28623.patch"},"body":"if we have 5 nodes cluster(such as 1,2,3,4,5 node),when 5 node leave the cluster,and now cluster delete one index.\r\nthen restart all nodes,if 1,2,5 join and find master,then 3,4 join the cluster.After started, this cluster state is red,\r\nbecause the delete index(lose some shards) is still part of the cluster metedata.\r\nto avoid this problem,this check the tomstones and indices,if tomston has index which is same in indices,\r\ndon't put index in metedata\r\n\r\n<!--\r\nThank you for your interest in and contributing to Elasticsearch! There\r\nare a few simple things to check before submitting your pull request\r\nthat can help with the review process. You should delete these items\r\nfrom your submission, but they are here to help bring them to your\r\nattention.\r\n-->\r\n\r\n- Have you signed the [contributor license agreement](https://www.elastic.co/contributor-agreement)?\r\n- Have you followed the [contributor guidelines](https://github.com/elastic/elasticsearch/blob/master/CONTRIBUTING.md)?\r\n- If submitting code, have you built your formula locally prior to submission with `gradle check`?\r\n- If submitting code, is your pull request against master? Unless there is a good reason otherwise, we prefer pull requests against master and will backport as needed.\r\n- If submitting code, have you checked that your submission is for an [OS that we support](https://www.elastic.co/support/matrix#show_os)?\r\n- If you are submitting this code for a class then read our [policy](https://github.com/elastic/elasticsearch/blob/master/CONTRIBUTING.md#contributing-as-part-of-a-class) for that.\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/371544236","html_url":"https://github.com/elastic/elasticsearch/pull/28623#issuecomment-371544236","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/28623","id":371544236,"user":{"login":"DaveCTurner","id":5058284,"avatar_url":"https://avatars3.githubusercontent.com/u/5058284?v=4","gravatar_id":"","url":"https://api.github.com/users/DaveCTurner","html_url":"https://github.com/DaveCTurner","followers_url":"https://api.github.com/users/DaveCTurner/followers","following_url":"https://api.github.com/users/DaveCTurner/following{/other_user}","gists_url":"https://api.github.com/users/DaveCTurner/gists{/gist_id}","starred_url":"https://api.github.com/users/DaveCTurner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DaveCTurner/subscriptions","organizations_url":"https://api.github.com/users/DaveCTurner/orgs","repos_url":"https://api.github.com/users/DaveCTurner/repos","events_url":"https://api.github.com/users/DaveCTurner/events{/privacy}","received_events_url":"https://api.github.com/users/DaveCTurner/received_events","type":"User","site_admin":false},"created_at":"2018-03-08T16:38:21Z","updated_at":"2018-03-08T16:38:21Z","author_association":"CONTRIBUTOR","body":"On further thought, I think the OP made the right choice: `Gateway` is the best place to fix this.\r\n\r\nUnfortunately, its correctness relies on a guarantee that doesn't currently hold: it is possible to commit a tombstone for an index but to fail to propagate that commit to the other master nodes, allowing the cluster to carry on using the index for an extended period, and this PR would mean that the index might then be deleted at the next full cluster restart. In the land of pure eventual consistency this is arguably the correct behaviour, but Elasticsearch currently offers slightly stronger guarantees about when things like index deletions occur, and I can imagine some users would find the new behaviour to be upsetting. In a situation where there's a conflict between the indices and the tombstones, we'd rather take the conservative path, leaving the data intact and deleting the tombstone.\r\n\r\nI'm marking this as \"won't fix\" because we won't be able to fix this in Elasticsearch 6.x or earlier. In fact it will eventually be fixed as a natural consequence of the changes we're making in the cluster state management area as part of Elasticsearch 7+, which will prevent there ever being a conflict between the sets of indices and tombstones.\r\n\r\nWe should, however, clean up any tombstones that conflict with indices, rather than leaving them in the cluster state."}}
{"actor":{"display_login":"DaveCTurner"}}