{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/20279","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/20279/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/20279/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/20279/events","html_url":"https://github.com/elastic/elasticsearch/issues/20279","id":174473812,"number":20279,"title":"TRA waits when an index doesn't exist but fails immediately when shard is not found","user":{"login":"bleskes","id":1006375,"avatar_url":"https://avatars.githubusercontent.com/u/1006375?v=3","gravatar_id":"","url":"https://api.github.com/users/bleskes","html_url":"https://github.com/bleskes","followers_url":"https://api.github.com/users/bleskes/followers","following_url":"https://api.github.com/users/bleskes/following{/other_user}","gists_url":"https://api.github.com/users/bleskes/gists{/gist_id}","starred_url":"https://api.github.com/users/bleskes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bleskes/subscriptions","organizations_url":"https://api.github.com/users/bleskes/orgs","repos_url":"https://api.github.com/users/bleskes/repos","events_url":"https://api.github.com/users/bleskes/events{/privacy}","received_events_url":"https://api.github.com/users/bleskes/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Core","name":":Core","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/adoptme","name":"adoptme","color":"207de5"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-09-01T09:11:48Z","updated_at":"2016-09-01T10:15:47Z","closed_at":null,"body":"TransportReplicationAction currently has an inconsistency in how it deals with requests that refer to things that don't exist (which is different than not available).\r\n\r\n1) When an index is not found in the cluster state, we go into a retry loop where we wait for the index to appear.\r\n2) When a request comes in for a shard that doesn't exists (i.e., the shard id is higher than the number of shards ) we fail immediately - as it will never appear.\r\n\r\nThis is surprising and we should fix it.\r\n\r\nIn my opinion we should:\r\n1) Require ReplicationRequests to have a complete ShardId when they get to the reroute phase in TRA.\r\n2) Fail immediately when that shard id can not be resolved.\r\n3) Change TransportIndexAction and similar write actions to resolve the incoming requests and set their proper shard id (with index uuid). If they need to create the index, they can go ahead, but then it's up to them to also wait until the current (data) node, knows about the index that was just created. We can have a shared utility method for this on `AutoCreateIndex`.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/244036895","html_url":"https://github.com/elastic/elasticsearch/issues/20279#issuecomment-244036895","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/20279","id":244036895,"user":{"login":"bleskes","id":1006375,"avatar_url":"https://avatars.githubusercontent.com/u/1006375?v=3","gravatar_id":"","url":"https://api.github.com/users/bleskes","html_url":"https://github.com/bleskes","followers_url":"https://api.github.com/users/bleskes/followers","following_url":"https://api.github.com/users/bleskes/following{/other_user}","gists_url":"https://api.github.com/users/bleskes/gists{/gist_id}","starred_url":"https://api.github.com/users/bleskes/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bleskes/subscriptions","organizations_url":"https://api.github.com/users/bleskes/orgs","repos_url":"https://api.github.com/users/bleskes/repos","events_url":"https://api.github.com/users/bleskes/events{/privacy}","received_events_url":"https://api.github.com/users/bleskes/received_events","type":"User","site_admin":false},"created_at":"2016-09-01T10:15:47Z","updated_at":"2016-09-01T10:15:47Z","body":"one more note - when converting an index name to an `Index` instances, we always have to wait in the case the index is not there. The reason is that, sadly,  a create index operation can finish successfully without all nodes knowing about it. Since we can then accept a follow up indexing request on arbitrary nodes, which should also give a chance for lagging cluster state to arrive. Practically this means that TransportIndexAction and friends need to wait also when they don't create a missing index . I expect this to be a common pattern when we move all index resolving to the rest layer (once the transport client has been removed, so there is time...)"}}
{"actor":{"display_login":"bleskes"}}