{"action":"created","issue":{"url":"https://api.github.com/repos/vespa-engine/vespa/issues/6009","repository_url":"https://api.github.com/repos/vespa-engine/vespa","labels_url":"https://api.github.com/repos/vespa-engine/vespa/issues/6009/labels{/name}","comments_url":"https://api.github.com/repos/vespa-engine/vespa/issues/6009/comments","events_url":"https://api.github.com/repos/vespa-engine/vespa/issues/6009/events","html_url":"https://github.com/vespa-engine/vespa/issues/6009","id":327720860,"node_id":"MDU6SXNzdWUzMjc3MjA4NjA=","number":6009,"title":"Ranking feature Emission from query tree clauses for matching ","user":{"login":"vandit-thakkar","id":35453933,"node_id":"MDQ6VXNlcjM1NDUzOTMz","avatar_url":"https://avatars1.githubusercontent.com/u/35453933?v=4","gravatar_id":"","url":"https://api.github.com/users/vandit-thakkar","html_url":"https://github.com/vandit-thakkar","followers_url":"https://api.github.com/users/vandit-thakkar/followers","following_url":"https://api.github.com/users/vandit-thakkar/following{/other_user}","gists_url":"https://api.github.com/users/vandit-thakkar/gists{/gist_id}","starred_url":"https://api.github.com/users/vandit-thakkar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vandit-thakkar/subscriptions","organizations_url":"https://api.github.com/users/vandit-thakkar/orgs","repos_url":"https://api.github.com/users/vandit-thakkar/repos","events_url":"https://api.github.com/users/vandit-thakkar/events{/privacy}","received_events_url":"https://api.github.com/users/vandit-thakkar/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-05-30T13:03:09Z","updated_at":"2018-06-07T14:27:21Z","closed_at":"2018-06-05T07:32:28Z","author_association":"NONE","body":"The ranking expression is formed with rank features and constants. http://docs.vespa.ai/documentation/reference/rank-features.html describes all the features we can use, we have` matches(<field_name>,<term_position>)` as a rank feature which returns 0/1. \r\n\r\nNow, can't we have `matches(label)` where label represents a label of a QueryTree item [ representing something like `( title : samsung )`] perhaps https://github.com/vespa-engine/vespa/blob/master/container-search/src/main/java/com/yahoo/prelude/query/BlockItem.java. \r\n\r\nWhile query evaluation, anyways, ( title : samsung ) checking happens for all documents, so can't this emit a rank feature, referenced by its label or anything which represents `title:samsung`. \r\n\r\nI believe while preparing any vespa application (vespa-deploy prepare), it's checked if a match feature's field name is in index environment. https://github.com/vespa-engine/vespa/blob/master/searchlib/src/vespa/searchlib/fef/parametervalidator.cpp, that's absolutely fine. So, if Vespa had `matches(label)` type feature, it can be checked only at query time that the field of (field, term) which that label refers to is in index environment or not. \r\n\r\nIf this doesn't compromise with Vespa design principles, please provide such feature such that hardcoding of field_name in ranking expression can be avoided. \r\n\r\n"},"comment":{"url":"https://api.github.com/repos/vespa-engine/vespa/issues/comments/395440798","html_url":"https://github.com/vespa-engine/vespa/issues/6009#issuecomment-395440798","issue_url":"https://api.github.com/repos/vespa-engine/vespa/issues/6009","id":395440798,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NTQ0MDc5OA==","user":{"login":"bratseth","id":16574012,"node_id":"MDQ6VXNlcjE2NTc0MDEy","avatar_url":"https://avatars2.githubusercontent.com/u/16574012?v=4","gravatar_id":"","url":"https://api.github.com/users/bratseth","html_url":"https://github.com/bratseth","followers_url":"https://api.github.com/users/bratseth/followers","following_url":"https://api.github.com/users/bratseth/following{/other_user}","gists_url":"https://api.github.com/users/bratseth/gists{/gist_id}","starred_url":"https://api.github.com/users/bratseth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bratseth/subscriptions","organizations_url":"https://api.github.com/users/bratseth/orgs","repos_url":"https://api.github.com/users/bratseth/repos","events_url":"https://api.github.com/users/bratseth/events{/privacy}","received_events_url":"https://api.github.com/users/bratseth/received_events","type":"User","site_admin":false},"created_at":"2018-06-07T14:27:21Z","updated_at":"2018-06-07T14:27:21Z","author_association":"MEMBER","body":"You are more or less correct, but a simpler way to put it: Your ranking expression is something like \"a + b + c + ...\" and this is compiled once, not once per query (that would be way too slow).\r\n\r\nSo, even if b and c (and so on) is 0 for a given query and that is only computed once, we still evaluate the full (compiled) expression \"a + b + c + ...\", and if everything else is cheap this dominates cost.\r\n\r\nThanks for the numbers. I'll sanity check the performance, but they seem reasonable.\r\n\r\nTwo remaining questions for you:\r\n- Do you need to match 40k documents per query on average _over the entire corpus_, or _per node_?\r\n- Did you consider just using \"match\" as the entire ranking expression?"}}
{"actor":{"display_login":"bratseth"}}