{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/18267","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18267/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18267/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18267/events","html_url":"https://github.com/elastic/elasticsearch/issues/18267","id":154189595,"number":18267,"title":"Remove user-configurable analyzer versions ","user":{"login":"clintongormley","id":56599,"avatar_url":"https://avatars.githubusercontent.com/u/56599?v=3","gravatar_id":"","url":"https://api.github.com/users/clintongormley","html_url":"https://github.com/clintongormley","followers_url":"https://api.github.com/users/clintongormley/followers","following_url":"https://api.github.com/users/clintongormley/following{/other_user}","gists_url":"https://api.github.com/users/clintongormley/gists{/gist_id}","starred_url":"https://api.github.com/users/clintongormley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clintongormley/subscriptions","organizations_url":"https://api.github.com/users/clintongormley/orgs","repos_url":"https://api.github.com/users/clintongormley/repos","events_url":"https://api.github.com/users/clintongormley/events{/privacy}","received_events_url":"https://api.github.com/users/clintongormley/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Analysis","name":":Analysis","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/adoptme","name":"adoptme","color":"207de5"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/breaking","name":"breaking","color":"ff0000"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/low%20hanging%20fruit","name":"low hanging fruit","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/v5.0.0","name":"v5.0.0","color":"dddddd"}],"state":"open","locked":false,"assignee":{"login":"MaineC","id":70953,"avatar_url":"https://avatars.githubusercontent.com/u/70953?v=3","gravatar_id":"","url":"https://api.github.com/users/MaineC","html_url":"https://github.com/MaineC","followers_url":"https://api.github.com/users/MaineC/followers","following_url":"https://api.github.com/users/MaineC/following{/other_user}","gists_url":"https://api.github.com/users/MaineC/gists{/gist_id}","starred_url":"https://api.github.com/users/MaineC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MaineC/subscriptions","organizations_url":"https://api.github.com/users/MaineC/orgs","repos_url":"https://api.github.com/users/MaineC/repos","events_url":"https://api.github.com/users/MaineC/events{/privacy}","received_events_url":"https://api.github.com/users/MaineC/received_events","type":"User","site_admin":false},"assignees":[{"login":"MaineC","id":70953,"avatar_url":"https://avatars.githubusercontent.com/u/70953?v=3","gravatar_id":"","url":"https://api.github.com/users/MaineC","html_url":"https://github.com/MaineC","followers_url":"https://api.github.com/users/MaineC/followers","following_url":"https://api.github.com/users/MaineC/following{/other_user}","gists_url":"https://api.github.com/users/MaineC/gists{/gist_id}","starred_url":"https://api.github.com/users/MaineC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MaineC/subscriptions","organizations_url":"https://api.github.com/users/MaineC/orgs","repos_url":"https://api.github.com/users/MaineC/repos","events_url":"https://api.github.com/users/MaineC/events{/privacy}","received_events_url":"https://api.github.com/users/MaineC/received_events","type":"User","site_admin":false}],"milestone":null,"comments":4,"created_at":"2016-05-11T08:40:31Z","updated_at":"2016-06-28T07:40:37Z","closed_at":null,"body":"All analyzers, tokenizers, etc accept a `version` parameter today (see https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis.html#backwards-compatibility).\r\n\r\nThe problem is, it doesn't work as expected.  For instance in master, you can specify that you want version `3.0` of an analyzer and your request will be accepted, but the code doesn't HAVE version `3.0` of the analyzer. Lucene 6 only includes version constants for 5.0 and above.\r\n\r\nInternally, the version parameter is useful as it means we can recognise an index created before an analyzer was changed (eg see [ArabicAnalyzer.java](https://github.com/apache/lucene-solr/blob/branch_6x/lucene/analysis/common/src/java/org/apache/lucene/analysis/ar/ArabicAnalyzer.java#L137)) but we can't read indices created before Lucene 5.0 anyway, so exposing the `version` in the API doesn't make much sense to me."},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/228975114","html_url":"https://github.com/elastic/elasticsearch/issues/18267#issuecomment-228975114","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/18267","id":228975114,"user":{"login":"MaineC","id":70953,"avatar_url":"https://avatars.githubusercontent.com/u/70953?v=3","gravatar_id":"","url":"https://api.github.com/users/MaineC","html_url":"https://github.com/MaineC","followers_url":"https://api.github.com/users/MaineC/followers","following_url":"https://api.github.com/users/MaineC/following{/other_user}","gists_url":"https://api.github.com/users/MaineC/gists{/gist_id}","starred_url":"https://api.github.com/users/MaineC/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MaineC/subscriptions","organizations_url":"https://api.github.com/users/MaineC/orgs","repos_url":"https://api.github.com/users/MaineC/repos","events_url":"https://api.github.com/users/MaineC/events{/privacy}","received_events_url":"https://api.github.com/users/MaineC/received_events","type":"User","site_admin":false},"created_at":"2016-06-28T07:40:37Z","updated_at":"2016-06-28T07:40:37Z","body":"> i don't think this should be user-configurable at all. It should be used purely to allow bwc for \r\n> existing indices (and that use is internal only)\r\n\r\nMakes sense to me. When looking for the code that actually parses the snippet you provided it looks to me like this is being checked only once the analyzer is actually used. Here's what I tried:\r\n\r\n    PUT foo\r\n    {\r\n      \"settings\": {\r\n        \"analysis\": {\r\n          \"analyzer\": {\r\n            \"old_standard\": {\r\n              \"type\": \"standard\",\r\n              \"version\": \"2.0\",\r\n              \"foobar\": \"23\"\r\n            }\r\n          }\r\n        }\r\n      }\r\n    }\r\n\r\nI can easily add this kind of setting without any complaints in the response nor in the logs. I can also use the analyzer with\r\n\r\n    GET /foo/_analyze\r\n    {\r\n      \"analyzer\" : \"old_standard\",\r\n      \"text\" : \"this is a test\"\r\n    }\r\n\r\nwithout issue.\r\n\r\n    DELETE /foo\r\n\r\n    PUT foo\r\n    {\r\n      \"settings\": {\r\n        \"analysis\": {\r\n          \"analyzer\": {\r\n            \"old_standard\": {\r\n              \"type\": \"standard\",\r\n              \"version\": \"bullshit\"\r\n            }\r\n          }\r\n        }\r\n      }\r\n    }\r\n\r\nThis can be set just fine. When trying to use the analyzer through the _analyze endpoint I get good looking analysis results through HTTP, but a \r\n\r\n    2016-06-28 09:28:36,102][WARN ][index.analysis           ] [Vidar] [foo] no version match bullshit,     default to 6.0.0\r\n    java.text.ParseException: Failed to parse major version from \"bullshit\" (got: bullshit)\r\n    \tat org.apache.lucene.util.Version.parse(Version.java:161)\r\n    \tat org.elasticsearch.common.lucene.Lucene.parseVersion(Lucene.java:112)\r\n    \tat org.elasticsearch.index.analysis.Analysis.parseAnalysisVersion(Ana\r\n\r\nin the logs.\r\n\r\nOthers may correct me if I'm wrong, but to me this looks a lot like analysis settings aren't validated (?!) here:\r\n\r\nhttps://github.com/elastic/elasticsearch/blob/93415d45061ca35042c21c45a1593cca39adb8d1/core/src/main/java/org/elasticsearch/common/settings/IndexScopedSettings.java#L153 \r\n\r\nNot sure how deep validation should go, explicitly disallowing the `version` entry but allowing other bullshit entries seems confusing to me."}}
{"actor":{"display_login":"MaineC"}}