{"action":"created","comment":{"url":"https://api.github.com/repos/alphagov/rummager/pulls/comments/52178381","id":52178381,"diff_hunk":"@@ -1,23 +1,27 @@\n+require \"query_components/user_filter\"\n+require \"query_components/visibility_filter\"\n+require 'elasticsearch/queries'\n+\n module QueryComponents\n   class Filter < BaseComponent\n+    include Elasticsearch::Queries\n+\n     def payload(excluding = [])\n-      rejects = []\n-      filters = []\n+      user_filter = QueryComponents::UserFilter.new(search_params)\n+      visibility_filter = QueryComponents::VisibilityFilter.new(search_params)\n \n-      param_filters = search_params.filters.reject do |filter|\n-        excluding.include?(filter.field_name)\n-      end\n+      filters = combine_filters(user_filter.filters_payload(excluding), :and)","path":"lib/query_components/filter.rb","position":18,"original_position":18,"commit_id":"8c1dd966190c40672cc0d711b6e9eee6a0c1e79b","original_commit_id":"8c1dd966190c40672cc0d711b6e9eee6a0c1e79b","user":{"login":"jackscotti","id":5038475,"avatar_url":"https://avatars.githubusercontent.com/u/5038475?v=3","gravatar_id":"","url":"https://api.github.com/users/jackscotti","html_url":"https://github.com/jackscotti","followers_url":"https://api.github.com/users/jackscotti/followers","following_url":"https://api.github.com/users/jackscotti/following{/other_user}","gists_url":"https://api.github.com/users/jackscotti/gists{/gist_id}","starred_url":"https://api.github.com/users/jackscotti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jackscotti/subscriptions","organizations_url":"https://api.github.com/users/jackscotti/orgs","repos_url":"https://api.github.com/users/jackscotti/repos","events_url":"https://api.github.com/users/jackscotti/events{/privacy}","received_events_url":"https://api.github.com/users/jackscotti/received_events","type":"User","site_admin":false},"body":"for consistency, please rewrite this with the same style you've used in the lines below","created_at":"2016-02-08T14:59:27Z","updated_at":"2016-02-08T14:59:27Z","html_url":"https://github.com/alphagov/rummager/pull/556#discussion_r52178381","pull_request_url":"https://api.github.com/repos/alphagov/rummager/pulls/556","_links":{"self":{"href":"https://api.github.com/repos/alphagov/rummager/pulls/comments/52178381"},"html":{"href":"https://github.com/alphagov/rummager/pull/556#discussion_r52178381"},"pull_request":{"href":"https://api.github.com/repos/alphagov/rummager/pulls/556"}}},"pull_request":{"url":"https://api.github.com/repos/alphagov/rummager/pulls/556","id":58458521,"html_url":"https://github.com/alphagov/rummager/pull/556","diff_url":"https://github.com/alphagov/rummager/pull/556.diff","patch_url":"https://github.com/alphagov/rummager/pull/556.patch","issue_url":"https://api.github.com/repos/alphagov/rummager/issues/556","number":556,"state":"open","locked":false,"title":"[WIP] Add a withdrawn field to all document types","user":{"login":"MatMoore","id":87579,"avatar_url":"https://avatars.githubusercontent.com/u/87579?v=3","gravatar_id":"","url":"https://api.github.com/users/MatMoore","html_url":"https://github.com/MatMoore","followers_url":"https://api.github.com/users/MatMoore/followers","following_url":"https://api.github.com/users/MatMoore/following{/other_user}","gists_url":"https://api.github.com/users/MatMoore/gists{/gist_id}","starred_url":"https://api.github.com/users/MatMoore/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MatMoore/subscriptions","organizations_url":"https://api.github.com/users/MatMoore/orgs","repos_url":"https://api.github.com/users/MatMoore/repos","events_url":"https://api.github.com/users/MatMoore/events{/privacy}","received_events_url":"https://api.github.com/users/MatMoore/received_events","type":"User","site_admin":false},"body":"This will allow us to include withdrawn documents when we do content audits.\r\nhttps://trello.com/c/RkrlGszq/473-keep-items-in-search-index-when-withdrawn-but-not-redirected\r\n\r\nIf `is_withdrawn` is set on a content item, it is not returned by searches, except if a debug flag is passed in.\r\n\r\nWhen we do a basic search, the elasticsearch query is built up in parts. I've refactored the `Filter` component so that it delegates to a `UserFilter` and a `VisibilityFilter`, where `VisibilityFilter` contains a query to exclude the withdrawn content.\r\n\r\n`UserFilter` is responsible for building up filters from the search params. There was a bunch of reusable code here that used to be private methods of this component. They all manipulate hashes that are serialised into JSON for the elasticsearch request. I've moved them out into a mixin in the elasticsearch module so I can reuse them in `Filter` and `VisibilityFilter`. If there is a better place for this let me know. I suspect there might be some duplication with the advanced search logic, which has its own builder class.\r\n\r\n`#filter_hash` is slightly different, because this operates on another class object representing a filter that is created in the search parameter parser. I've left this as a private method of the UserFilter for now but it may make sense to move this method to the filter object itself.\r\n\r\nI still need to have a look at advanced searches as they are handled differently.","created_at":"2016-02-05T17:25:51Z","updated_at":"2016-02-08T14:59:27Z","closed_at":null,"merged_at":null,"merge_commit_sha":"5c7a7de259efb3f72bffd60a60924666812019b7","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/alphagov/rummager/pulls/556/commits","review_comments_url":"https://api.github.com/repos/alphagov/rummager/pulls/556/comments","review_comment_url":"https://api.github.com/repos/alphagov/rummager/pulls/comments{/number}","comments_url":"https://api.github.com/repos/alphagov/rummager/issues/556/comments","statuses_url":"https://api.github.com/repos/alphagov/rummager/statuses/8c1dd966190c40672cc0d711b6e9eee6a0c1e79b","head":{"label":"alphagov:add-withdrawn-to-index","ref":"add-withdrawn-to-index","sha":"8c1dd966190c40672cc0d711b6e9eee6a0c1e79b","user":{"login":"alphagov","id":596977,"avatar_url":"https://avatars.githubusercontent.com/u/596977?v=3","gravatar_id":"","url":"https://api.github.com/users/alphagov","html_url":"https://github.com/alphagov","followers_url":"https://api.github.com/users/alphagov/followers","following_url":"https://api.github.com/users/alphagov/following{/other_user}","gists_url":"https://api.github.com/users/alphagov/gists{/gist_id}","starred_url":"https://api.github.com/users/alphagov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alphagov/subscriptions","organizations_url":"https://api.github.com/users/alphagov/orgs","repos_url":"https://api.github.com/users/alphagov/repos","events_url":"https://api.github.com/users/alphagov/events{/privacy}","received_events_url":"https://api.github.com/users/alphagov/received_events","type":"Organization","site_admin":false},"repo":{"id":2686767,"name":"rummager","full_name":"alphagov/rummager","owner":{"login":"alphagov","id":596977,"avatar_url":"https://avatars.githubusercontent.com/u/596977?v=3","gravatar_id":"","url":"https://api.github.com/users/alphagov","html_url":"https://github.com/alphagov","followers_url":"https://api.github.com/users/alphagov/followers","following_url":"https://api.github.com/users/alphagov/following{/other_user}","gists_url":"https://api.github.com/users/alphagov/gists{/gist_id}","starred_url":"https://api.github.com/users/alphagov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alphagov/subscriptions","organizations_url":"https://api.github.com/users/alphagov/orgs","repos_url":"https://api.github.com/users/alphagov/repos","events_url":"https://api.github.com/users/alphagov/events{/privacy}","received_events_url":"https://api.github.com/users/alphagov/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/alphagov/rummager","description":"Rummager is the internal GOV.UK API for search.","fork":false,"url":"https://api.github.com/repos/alphagov/rummager","forks_url":"https://api.github.com/repos/alphagov/rummager/forks","keys_url":"https://api.github.com/repos/alphagov/rummager/keys{/key_id}","collaborators_url":"https://api.github.com/repos/alphagov/rummager/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/alphagov/rummager/teams","hooks_url":"https://api.github.com/repos/alphagov/rummager/hooks","issue_events_url":"https://api.github.com/repos/alphagov/rummager/issues/events{/number}","events_url":"https://api.github.com/repos/alphagov/rummager/events","assignees_url":"https://api.github.com/repos/alphagov/rummager/assignees{/user}","branches_url":"https://api.github.com/repos/alphagov/rummager/branches{/branch}","tags_url":"https://api.github.com/repos/alphagov/rummager/tags","blobs_url":"https://api.github.com/repos/alphagov/rummager/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/alphagov/rummager/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/alphagov/rummager/git/refs{/sha}","trees_url":"https://api.github.com/repos/alphagov/rummager/git/trees{/sha}","statuses_url":"https://api.github.com/repos/alphagov/rummager/statuses/{sha}","languages_url":"https://api.github.com/repos/alphagov/rummager/languages","stargazers_url":"https://api.github.com/repos/alphagov/rummager/stargazers","contributors_url":"https://api.github.com/repos/alphagov/rummager/contributors","subscribers_url":"https://api.github.com/repos/alphagov/rummager/subscribers","subscription_url":"https://api.github.com/repos/alphagov/rummager/subscription","commits_url":"https://api.github.com/repos/alphagov/rummager/commits{/sha}","git_commits_url":"https://api.github.com/repos/alphagov/rummager/git/commits{/sha}","comments_url":"https://api.github.com/repos/alphagov/rummager/comments{/number}","issue_comment_url":"https://api.github.com/repos/alphagov/rummager/issues/comments{/number}","contents_url":"https://api.github.com/repos/alphagov/rummager/contents/{+path}","compare_url":"https://api.github.com/repos/alphagov/rummager/compare/{base}...{head}","merges_url":"https://api.github.com/repos/alphagov/rummager/merges","archive_url":"https://api.github.com/repos/alphagov/rummager/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/alphagov/rummager/downloads","issues_url":"https://api.github.com/repos/alphagov/rummager/issues{/number}","pulls_url":"https://api.github.com/repos/alphagov/rummager/pulls{/number}","milestones_url":"https://api.github.com/repos/alphagov/rummager/milestones{/number}","notifications_url":"https://api.github.com/repos/alphagov/rummager/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/alphagov/rummager/labels{/name}","releases_url":"https://api.github.com/repos/alphagov/rummager/releases{/id}","deployments_url":"https://api.github.com/repos/alphagov/rummager/deployments","created_at":"2011-11-01T10:12:45Z","updated_at":"2015-11-05T15:02:01Z","pushed_at":"2016-02-08T14:38:42Z","git_url":"git://github.com/alphagov/rummager.git","ssh_url":"git@github.com:alphagov/rummager.git","clone_url":"https://github.com/alphagov/rummager.git","svn_url":"https://github.com/alphagov/rummager","homepage":"https://www.gov.uk/api/search.json?q=","size":3795,"stargazers_count":28,"watchers_count":28,"language":"Ruby","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":8,"mirror_url":null,"open_issues_count":2,"forks":8,"open_issues":2,"watchers":28,"default_branch":"master"}},"base":{"label":"alphagov:master","ref":"master","sha":"b224be9b0f4691933f6fe4282a87a2d9b1bd4575","user":{"login":"alphagov","id":596977,"avatar_url":"https://avatars.githubusercontent.com/u/596977?v=3","gravatar_id":"","url":"https://api.github.com/users/alphagov","html_url":"https://github.com/alphagov","followers_url":"https://api.github.com/users/alphagov/followers","following_url":"https://api.github.com/users/alphagov/following{/other_user}","gists_url":"https://api.github.com/users/alphagov/gists{/gist_id}","starred_url":"https://api.github.com/users/alphagov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alphagov/subscriptions","organizations_url":"https://api.github.com/users/alphagov/orgs","repos_url":"https://api.github.com/users/alphagov/repos","events_url":"https://api.github.com/users/alphagov/events{/privacy}","received_events_url":"https://api.github.com/users/alphagov/received_events","type":"Organization","site_admin":false},"repo":{"id":2686767,"name":"rummager","full_name":"alphagov/rummager","owner":{"login":"alphagov","id":596977,"avatar_url":"https://avatars.githubusercontent.com/u/596977?v=3","gravatar_id":"","url":"https://api.github.com/users/alphagov","html_url":"https://github.com/alphagov","followers_url":"https://api.github.com/users/alphagov/followers","following_url":"https://api.github.com/users/alphagov/following{/other_user}","gists_url":"https://api.github.com/users/alphagov/gists{/gist_id}","starred_url":"https://api.github.com/users/alphagov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alphagov/subscriptions","organizations_url":"https://api.github.com/users/alphagov/orgs","repos_url":"https://api.github.com/users/alphagov/repos","events_url":"https://api.github.com/users/alphagov/events{/privacy}","received_events_url":"https://api.github.com/users/alphagov/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/alphagov/rummager","description":"Rummager is the internal GOV.UK API for search.","fork":false,"url":"https://api.github.com/repos/alphagov/rummager","forks_url":"https://api.github.com/repos/alphagov/rummager/forks","keys_url":"https://api.github.com/repos/alphagov/rummager/keys{/key_id}","collaborators_url":"https://api.github.com/repos/alphagov/rummager/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/alphagov/rummager/teams","hooks_url":"https://api.github.com/repos/alphagov/rummager/hooks","issue_events_url":"https://api.github.com/repos/alphagov/rummager/issues/events{/number}","events_url":"https://api.github.com/repos/alphagov/rummager/events","assignees_url":"https://api.github.com/repos/alphagov/rummager/assignees{/user}","branches_url":"https://api.github.com/repos/alphagov/rummager/branches{/branch}","tags_url":"https://api.github.com/repos/alphagov/rummager/tags","blobs_url":"https://api.github.com/repos/alphagov/rummager/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/alphagov/rummager/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/alphagov/rummager/git/refs{/sha}","trees_url":"https://api.github.com/repos/alphagov/rummager/git/trees{/sha}","statuses_url":"https://api.github.com/repos/alphagov/rummager/statuses/{sha}","languages_url":"https://api.github.com/repos/alphagov/rummager/languages","stargazers_url":"https://api.github.com/repos/alphagov/rummager/stargazers","contributors_url":"https://api.github.com/repos/alphagov/rummager/contributors","subscribers_url":"https://api.github.com/repos/alphagov/rummager/subscribers","subscription_url":"https://api.github.com/repos/alphagov/rummager/subscription","commits_url":"https://api.github.com/repos/alphagov/rummager/commits{/sha}","git_commits_url":"https://api.github.com/repos/alphagov/rummager/git/commits{/sha}","comments_url":"https://api.github.com/repos/alphagov/rummager/comments{/number}","issue_comment_url":"https://api.github.com/repos/alphagov/rummager/issues/comments{/number}","contents_url":"https://api.github.com/repos/alphagov/rummager/contents/{+path}","compare_url":"https://api.github.com/repos/alphagov/rummager/compare/{base}...{head}","merges_url":"https://api.github.com/repos/alphagov/rummager/merges","archive_url":"https://api.github.com/repos/alphagov/rummager/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/alphagov/rummager/downloads","issues_url":"https://api.github.com/repos/alphagov/rummager/issues{/number}","pulls_url":"https://api.github.com/repos/alphagov/rummager/pulls{/number}","milestones_url":"https://api.github.com/repos/alphagov/rummager/milestones{/number}","notifications_url":"https://api.github.com/repos/alphagov/rummager/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/alphagov/rummager/labels{/name}","releases_url":"https://api.github.com/repos/alphagov/rummager/releases{/id}","deployments_url":"https://api.github.com/repos/alphagov/rummager/deployments","created_at":"2011-11-01T10:12:45Z","updated_at":"2015-11-05T15:02:01Z","pushed_at":"2016-02-08T14:38:42Z","git_url":"git://github.com/alphagov/rummager.git","ssh_url":"git@github.com:alphagov/rummager.git","clone_url":"https://github.com/alphagov/rummager.git","svn_url":"https://github.com/alphagov/rummager","homepage":"https://www.gov.uk/api/search.json?q=","size":3795,"stargazers_count":28,"watchers_count":28,"language":"Ruby","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":8,"mirror_url":null,"open_issues_count":2,"forks":8,"open_issues":2,"watchers":28,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/alphagov/rummager/pulls/556"},"html":{"href":"https://github.com/alphagov/rummager/pull/556"},"issue":{"href":"https://api.github.com/repos/alphagov/rummager/issues/556"},"comments":{"href":"https://api.github.com/repos/alphagov/rummager/issues/556/comments"},"review_comments":{"href":"https://api.github.com/repos/alphagov/rummager/pulls/556/comments"},"review_comment":{"href":"https://api.github.com/repos/alphagov/rummager/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/alphagov/rummager/pulls/556/commits"},"statuses":{"href":"https://api.github.com/repos/alphagov/rummager/statuses/8c1dd966190c40672cc0d711b6e9eee6a0c1e79b"}}}}