{"action":"created","issue":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/20925","repository_url":"https://api.github.com/repos/elastic/elasticsearch","labels_url":"https://api.github.com/repos/elastic/elasticsearch/issues/20925/labels{/name}","comments_url":"https://api.github.com/repos/elastic/elasticsearch/issues/20925/comments","events_url":"https://api.github.com/repos/elastic/elasticsearch/issues/20925/events","html_url":"https://github.com/elastic/elasticsearch/pull/20925","id":182904450,"number":20925,"title":"Introduce a new \"all\" query that constructs \"_all\"-like queries","user":{"login":"dakrone","id":19060,"avatar_url":"https://avatars.githubusercontent.com/u/19060?v=3","gravatar_id":"","url":"https://api.github.com/users/dakrone","html_url":"https://github.com/dakrone","followers_url":"https://api.github.com/users/dakrone/followers","following_url":"https://api.github.com/users/dakrone/following{/other_user}","gists_url":"https://api.github.com/users/dakrone/gists{/gist_id}","starred_url":"https://api.github.com/users/dakrone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dakrone/subscriptions","organizations_url":"https://api.github.com/users/dakrone/orgs","repos_url":"https://api.github.com/users/dakrone/repos","events_url":"https://api.github.com/users/dakrone/events{/privacy}","received_events_url":"https://api.github.com/users/dakrone/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/:Query%20DSL","name":":Query DSL","color":"009800"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/discuss","name":"discuss","color":"fbca04"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/feature","name":"feature","color":"328a58"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/v5.1.0","name":"v5.1.0","color":"dddddd"},{"url":"https://api.github.com/repos/elastic/elasticsearch/labels/v6.0.0","name":"v6.0.0","color":"dddddd"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":17,"created_at":"2016-10-13T21:22:55Z","updated_at":"2016-10-18T07:15:23Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/elastic/elasticsearch/pulls/20925","html_url":"https://github.com/elastic/elasticsearch/pull/20925","diff_url":"https://github.com/elastic/elasticsearch/pull/20925.diff","patch_url":"https://github.com/elastic/elasticsearch/pull/20925.patch"},"body":"This commit introduces a new \"all\" query, which is intended down the\r\nroad to be a replacement for the current `_all` field.\r\n\r\nThe query is currently very rudimentary, it is intended to be a starting\r\npoint on which additional features can be added as needed. It currently\r\nlooks like:\r\n\r\n```json\r\nGET /_search\r\n{\r\n  \"query\": {\r\n    \"all\": {\r\n      \"query\": \"foo Bar baz\"\r\n    }\r\n  }\r\n}\r\n```\r\n\r\nInternally, what happens is that the \"all\" query analyzes the text using\r\na Lucene analyzer (by default, the `whitespace` analyzer) to split it\r\ninto tokens, so in this case `[\"foo\", \"Bar\", \"baz\"]` are the tokens\r\ngenerated. It then goes through all the supported types in the index's\r\nmappings, generating a `multi_match` query across all supported types\r\nfor each term, using `cross_fields`. As an example, pretend an index had\r\nthree fields, \"text-field\", \"num-field\", and \"date-field\". The above\r\n\"all\" query would generate a query that would look like:\r\n\r\n```json\r\n{\r\n  \"query\": {\r\n    \"constant_score\": {\r\n      \"filter\": {\r\n        \"bool\": {\r\n          \"should\": [\r\n            {\r\n              \"multi_match\": {\r\n                \"query\": \"foo\",\r\n                \"fields\": [\"text-field\", \"num-field\", \"date-field\"],\r\n                \"type\": \"cross_fields\",\r\n                \"lenient\": true\r\n              }\r\n            },\r\n            {\r\n              \"multi_match\": {\r\n                \"query\": \"Bar\",\r\n                \"fields\": [\"text-field\", \"num-field\", \"date-field\"],\r\n                \"type\": \"cross_fields\",\r\n                \"lenient\": true\r\n              }\r\n            },\r\n            {\r\n              \"multi_match\": {\r\n                \"query\": \"baz\",\r\n                \"fields\": [\"text-field\", \"num-field\", \"date-field\"],\r\n                \"type\": \"cross_fields\",\r\n                \"lenient\": true\r\n              }\r\n            }\r\n          ]\r\n        }\r\n      }\r\n    }\r\n  }\r\n}\r\n```\r\n\r\nNote that this has a few advantages over a regular query on the `_all`\r\nfield:\r\n\r\n- Tokens can be analyzed with their respective field's analyzers\r\n\r\nSo, if the \"text-field\" field used ngrams, each token could then be\r\nanalyzed with ngrams, so a document with \"foo\" in the \"text-field\" field\r\nusing bigrams would match an \"all\" query for \"foa\".\r\n\r\n- No pre-setup is required\r\n\r\nNothing stops someone from immediately using this with their data,\r\nno-reindexing required.\r\n\r\n- Leniency regarding mismatched field types\r\n\r\nThe reason why the leniency flag is used, is so a user could potentially\r\nuse this as a replacement for the Kibana-esque filter bar, where a user\r\nmight type:\r\n\r\n```\r\nproduction 200 10.29.2.1\r\n```\r\n\r\nExpecting to find documents for their production environment where a 200\r\nresponse code is returned from the given IP address. Since the tokens\r\nare treated individually, all numeric fields (int, long) can be searched\r\nfor 200, all the IP address fields can be searched for 10.29.2.1, and\r\nall text-like (string, text, keyword) fields can be searched for\r\n\"production\". This can closely mimic search the `_all` field using the\r\ngiven terms.\r\n\r\n- Compatible with existing `_all` field\r\n\r\nThe \"all\" query ignores fields starting with `_`, so this is something a\r\nuser can upgrade, start the transition towards, and then eventually move\r\naway from `_all` gracefully.\r\n\r\nThe \"all\" query also supports an `analyzer` parameter, to change the\r\nanalyzer used to separate the tokens. This is so a user could select\r\nsomething like the `keyword` analyzer to stop tokens from being split at\r\nwhitespace.\r\n\r\nRelates to #19784\r\n\r\n-----------------\r\n\r\nFurther work down the road:\r\n\r\nDown the road, what would be very nice is to be able to rewrite a query\r\nlike:\r\n\r\n```json\r\n{\r\n  \"query\": {\r\n    \"query_string\": {\r\n      \"query\": \"foo bar title:baz body:eggplant\"\r\n    }\r\n  }\r\n}\r\n```\r\n\r\nInto a query like:\r\n\r\n```json\r\n{\r\n  \"query\": {\r\n    \"bool\": {\r\n      \"should\": [\r\n        {\r\n          \"query_string\": {\r\n            \"query\": \"title:baz body:eggplant\"\r\n          }\r\n        },\r\n        {\r\n          \"all\": {\r\n            \"query\": \"foo bar\"\r\n          }\r\n        }\r\n      ]\r\n    }\r\n  }\r\n}\r\n```\r\n\r\nHowever, this is not currently part of the implementation and is only an\r\nidea of how this *could* be built upon."},"comment":{"url":"https://api.github.com/repos/elastic/elasticsearch/issues/comments/254426071","html_url":"https://github.com/elastic/elasticsearch/pull/20925#issuecomment-254426071","issue_url":"https://api.github.com/repos/elastic/elasticsearch/issues/20925","id":254426071,"user":{"login":"clintongormley","id":56599,"avatar_url":"https://avatars.githubusercontent.com/u/56599?v=3","gravatar_id":"","url":"https://api.github.com/users/clintongormley","html_url":"https://github.com/clintongormley","followers_url":"https://api.github.com/users/clintongormley/followers","following_url":"https://api.github.com/users/clintongormley/following{/other_user}","gists_url":"https://api.github.com/users/clintongormley/gists{/gist_id}","starred_url":"https://api.github.com/users/clintongormley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clintongormley/subscriptions","organizations_url":"https://api.github.com/users/clintongormley/orgs","repos_url":"https://api.github.com/users/clintongormley/repos","events_url":"https://api.github.com/users/clintongormley/events{/privacy}","received_events_url":"https://api.github.com/users/clintongormley/received_events","type":"User","site_admin":false},"created_at":"2016-10-18T07:15:23Z","updated_at":"2016-10-18T07:15:23Z","body":"> I think we're conflating two different use cases. I would argue that for the users that are advanced enough to use prefixes, wildcards, regexp, fuzzy, etc that the \"all\" query is not for them. These users can use the : syntax to specify which fields they want to match on in their query.\r\n\r\nThink about Kibana users.  Sure, some use the search bar just for filtering.  Others use it with at least ranges, wildcards, AND|OR|NOT etc.  Others use a lot more.  Either way, there's only one search field exposed in Kibana. So either we're exposing the query string query, or we're coming up with a new query parser syntax that can be backed by this _all query.  "}}
{"actor":{"display_login":"clintongormley"}}