{"action":"reopened","number":19389,"pull_request":{"url":"https://api.github.com/repos/docker/docker/pulls/19389","id":56257895,"html_url":"https://github.com/docker/docker/pull/19389","diff_url":"https://github.com/docker/docker/pull/19389.diff","patch_url":"https://github.com/docker/docker/pull/19389.patch","issue_url":"https://api.github.com/repos/docker/docker/issues/19389","number":19389,"state":"open","locked":false,"title":"[WIP] POC for a query language for filtering ps output (#17569)","user":{"login":"jawher","id":96384,"avatar_url":"https://avatars.githubusercontent.com/u/96384?v=3","gravatar_id":"","url":"https://api.github.com/users/jawher","html_url":"https://github.com/jawher","followers_url":"https://api.github.com/users/jawher/followers","following_url":"https://api.github.com/users/jawher/following{/other_user}","gists_url":"https://api.github.com/users/jawher/gists{/gist_id}","starred_url":"https://api.github.com/users/jawher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jawher/subscriptions","organizations_url":"https://api.github.com/users/jawher/orgs","repos_url":"https://api.github.com/users/jawher/repos","events_url":"https://api.github.com/users/jawher/events{/privacy}","received_events_url":"https://api.github.com/users/jawher/received_events","type":"User","site_admin":false},"body":"Feature proposal: #17569\r\nPlease head to the linked issue for more details on the query language syntax and for a preliminary discussion on this proposal.\r\n\r\n### This is just a quick and dirty POC\r\nwith the sole goal of having a working example to build the discussion on.\r\n\r\n## Modifications\r\n\r\n* Add a new top-level package in docker named `query`. This package parses a given query from a string to an AST. The AST can be used to filter containers, images, volumes, networks and any other list of items according to the query. The only requirement is for the item type to implement an interface (`query.Queryable`)\r\n* Make `containers/Container` implement `query.Queryable`\r\n* Add a `--query` flag to `docker ps`\r\n* Modify `engine-api/client/type/ContainerListOptions` to add a `Query` field (in a true quick and dirty fashion, I modified the concerned files in the vendor directory, but a proper PR to the engine-api project will be created if this proposal is accepted)\r\n* Modify `daemon/list.go` to use the provided query instead of psFilters if it is present. The current implementation is, again, a quick and dirty one, and is incomplete, i.e. doesn't handle all psFilters fields among other things.\r\n\r\n## Query language\r\n\r\nMore details in [bateau's project page](https://github.com/jawher/bateau)\r\n### Boolean fields\r\n\r\n```\r\nrunning\r\n```\r\n\r\n### Valued fields \r\ne.g. string, []string, int, duration, size, etc.\r\nSyntax: `field operator value`\r\nWhere operator is one of:\r\n* `=` for strict equality, `!=` for strict inequality\r\n* `~` for string contains (LIKE), `!~` for the negated version\r\n* `>`, `>=`, `<` and `<=` for numeric comparisons\r\n\r\nexamples:\r\n```\r\nexit=1\r\n```\r\n\r\n```\r\nname=server_1\r\n```\r\n```\r\nname~\"server\"\r\n```\r\n```\r\ncreated > 2h\r\n```\r\n\r\n### Negation\r\nAny clause can be negated using the `!` operator.\r\nExamples:\r\n```\r\n!running\r\n```\r\n```\r\n!(exit=1) // though this is equivalent to exit!=1\r\n```\r\n\r\n### Anding, Oring\r\n2 or more clauses can be combined using anding `&` or bring `|`:\r\n\r\n```\r\nrunning & created > 2d // running containers created more than 2 days ago\r\n```\r\n```\r\nsize>300MB & (created > 2M | docker_version~1.5 | docker_version~1.6) \r\n```\r\n```\r\n!label.arch | label.arch=x86 // containers without an \"arch\" label or with one set to \"x86\"\r\n```\r\n\r\n## Open questions\r\n\r\nDo you think this proposal has a chance of being accepted ? If the answer is yes, I'm willing to rework this PR to handle:\r\n\r\n* at least the current supported filters, if not more fields\r\n* generalise it to work on images, volumes and networks (am I forgetting any other item list in docker ?)\r\n\r\n### The query syntax:\r\nI reused the code I wrote for [bateau](https://github.com/jawher/bateau).\r\n\r\nWhat is the preferred syntax between:\r\n\r\n```\r\n!label.arch | label.arch=x86\r\nsize>300MB & (created > 2M | docker_version~1.5 | docker_version~1.6) \r\n```\r\n\r\nand\r\n```\r\nnot label.arch and label.arch=x86\r\nsize>300MB and (created > 2M and docker_version~1.5 or docker_version~1.6) \r\n```\r\n\r\n### The query parsing complexity\r\n\r\nOne of the points raised by @duglin in the original issue (#17569) was the cost of adding and maintaining the parsing code.\r\n\r\nThe current implementation is comprised of a lexer (200 LOC) and a recursive descent parser (~180 LOC).\r\nI also have a [working prototype with a lexer-less parser](https://gist.github.com/jawher/56ed7ec4ab8ea7e9d5e7) (~240LOC), which I believe will be easier to extend with new operators for example than trying to do so with the current lexer.\r\n\r\nAlso, the syntax as such should not need to be modified frequently, if at all.\r\nA more frequent task would be to add more fields to filter on.\r\nThis task can be be accomplished without touching the parsing logic: see `container/query.go` for how the list of fields of a containers are defined.\r\n\r\n### Interaction with the current ps filtering mechanism\r\n\r\nThe `-f`, `--since` and `--before` filters are a subset of the query language of this proposal.\r\n\r\nIf this proposal is accepted, one possibility would be to deprecate these 3 flags, and make them exclusive with `--query`:\r\n\r\n```\r\nUsage:\r\ndocker ps ([-f] [--since] [--before]) | --query)\r\n```\r\n\r\n### Implementation details\r\n\r\nWhat do you think about how `query.Queryable`  is directly implemented by the concerned types (Container for example) ?\r\nThe other possibility would be to created adapter types which would implement the iface in the `daemon` package, e.g. `containerQueryable`, etc.\r\n","created_at":"2016-01-16T17:07:51Z","updated_at":"2016-01-16T17:20:55Z","closed_at":null,"merged_at":null,"merge_commit_sha":"f6779a6f318b2dd813b10f13a0617adf1421ab73","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/docker/docker/pulls/19389/commits","review_comments_url":"https://api.github.com/repos/docker/docker/pulls/19389/comments","review_comment_url":"https://api.github.com/repos/docker/docker/pulls/comments{/number}","comments_url":"https://api.github.com/repos/docker/docker/issues/19389/comments","statuses_url":"https://api.github.com/repos/docker/docker/statuses/11409636dc4d18f2f018d7ac85a5f76c979e78e7","head":{"label":"jawher:17569-container-image-filtering-query-language","ref":"17569-container-image-filtering-query-language","sha":"11409636dc4d18f2f018d7ac85a5f76c979e78e7","user":{"login":"jawher","id":96384,"avatar_url":"https://avatars.githubusercontent.com/u/96384?v=3","gravatar_id":"","url":"https://api.github.com/users/jawher","html_url":"https://github.com/jawher","followers_url":"https://api.github.com/users/jawher/followers","following_url":"https://api.github.com/users/jawher/following{/other_user}","gists_url":"https://api.github.com/users/jawher/gists{/gist_id}","starred_url":"https://api.github.com/users/jawher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jawher/subscriptions","organizations_url":"https://api.github.com/users/jawher/orgs","repos_url":"https://api.github.com/users/jawher/repos","events_url":"https://api.github.com/users/jawher/events{/privacy}","received_events_url":"https://api.github.com/users/jawher/received_events","type":"User","site_admin":false},"repo":{"id":49780668,"name":"docker","full_name":"jawher/docker","owner":{"login":"jawher","id":96384,"avatar_url":"https://avatars.githubusercontent.com/u/96384?v=3","gravatar_id":"","url":"https://api.github.com/users/jawher","html_url":"https://github.com/jawher","followers_url":"https://api.github.com/users/jawher/followers","following_url":"https://api.github.com/users/jawher/following{/other_user}","gists_url":"https://api.github.com/users/jawher/gists{/gist_id}","starred_url":"https://api.github.com/users/jawher/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jawher/subscriptions","organizations_url":"https://api.github.com/users/jawher/orgs","repos_url":"https://api.github.com/users/jawher/repos","events_url":"https://api.github.com/users/jawher/events{/privacy}","received_events_url":"https://api.github.com/users/jawher/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/jawher/docker","description":"Docker - the open-source application container engine","fork":true,"url":"https://api.github.com/repos/jawher/docker","forks_url":"https://api.github.com/repos/jawher/docker/forks","keys_url":"https://api.github.com/repos/jawher/docker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/jawher/docker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/jawher/docker/teams","hooks_url":"https://api.github.com/repos/jawher/docker/hooks","issue_events_url":"https://api.github.com/repos/jawher/docker/issues/events{/number}","events_url":"https://api.github.com/repos/jawher/docker/events","assignees_url":"https://api.github.com/repos/jawher/docker/assignees{/user}","branches_url":"https://api.github.com/repos/jawher/docker/branches{/branch}","tags_url":"https://api.github.com/repos/jawher/docker/tags","blobs_url":"https://api.github.com/repos/jawher/docker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/jawher/docker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/jawher/docker/git/refs{/sha}","trees_url":"https://api.github.com/repos/jawher/docker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/jawher/docker/statuses/{sha}","languages_url":"https://api.github.com/repos/jawher/docker/languages","stargazers_url":"https://api.github.com/repos/jawher/docker/stargazers","contributors_url":"https://api.github.com/repos/jawher/docker/contributors","subscribers_url":"https://api.github.com/repos/jawher/docker/subscribers","subscription_url":"https://api.github.com/repos/jawher/docker/subscription","commits_url":"https://api.github.com/repos/jawher/docker/commits{/sha}","git_commits_url":"https://api.github.com/repos/jawher/docker/git/commits{/sha}","comments_url":"https://api.github.com/repos/jawher/docker/comments{/number}","issue_comment_url":"https://api.github.com/repos/jawher/docker/issues/comments{/number}","contents_url":"https://api.github.com/repos/jawher/docker/contents/{+path}","compare_url":"https://api.github.com/repos/jawher/docker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/jawher/docker/merges","archive_url":"https://api.github.com/repos/jawher/docker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/jawher/docker/downloads","issues_url":"https://api.github.com/repos/jawher/docker/issues{/number}","pulls_url":"https://api.github.com/repos/jawher/docker/pulls{/number}","milestones_url":"https://api.github.com/repos/jawher/docker/milestones{/number}","notifications_url":"https://api.github.com/repos/jawher/docker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/jawher/docker/labels{/name}","releases_url":"https://api.github.com/repos/jawher/docker/releases{/id}","created_at":"2016-01-16T16:14:49Z","updated_at":"2016-01-16T16:15:00Z","pushed_at":"2016-01-16T17:16:54Z","git_url":"git://github.com/jawher/docker.git","ssh_url":"git@github.com:jawher/docker.git","clone_url":"https://github.com/jawher/docker.git","svn_url":"https://github.com/jawher/docker","homepage":"http://www.docker.com","size":80164,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"docker:master","ref":"master","sha":"82401a4b13c0581908d46dbc3e7b475c7de48334","user":{"login":"docker","id":5429470,"avatar_url":"https://avatars.githubusercontent.com/u/5429470?v=3","gravatar_id":"","url":"https://api.github.com/users/docker","html_url":"https://github.com/docker","followers_url":"https://api.github.com/users/docker/followers","following_url":"https://api.github.com/users/docker/following{/other_user}","gists_url":"https://api.github.com/users/docker/gists{/gist_id}","starred_url":"https://api.github.com/users/docker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/docker/subscriptions","organizations_url":"https://api.github.com/users/docker/orgs","repos_url":"https://api.github.com/users/docker/repos","events_url":"https://api.github.com/users/docker/events{/privacy}","received_events_url":"https://api.github.com/users/docker/received_events","type":"Organization","site_admin":false},"repo":{"id":7691631,"name":"docker","full_name":"docker/docker","owner":{"login":"docker","id":5429470,"avatar_url":"https://avatars.githubusercontent.com/u/5429470?v=3","gravatar_id":"","url":"https://api.github.com/users/docker","html_url":"https://github.com/docker","followers_url":"https://api.github.com/users/docker/followers","following_url":"https://api.github.com/users/docker/following{/other_user}","gists_url":"https://api.github.com/users/docker/gists{/gist_id}","starred_url":"https://api.github.com/users/docker/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/docker/subscriptions","organizations_url":"https://api.github.com/users/docker/orgs","repos_url":"https://api.github.com/users/docker/repos","events_url":"https://api.github.com/users/docker/events{/privacy}","received_events_url":"https://api.github.com/users/docker/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/docker/docker","description":"Docker - the open-source application container engine","fork":false,"url":"https://api.github.com/repos/docker/docker","forks_url":"https://api.github.com/repos/docker/docker/forks","keys_url":"https://api.github.com/repos/docker/docker/keys{/key_id}","collaborators_url":"https://api.github.com/repos/docker/docker/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/docker/docker/teams","hooks_url":"https://api.github.com/repos/docker/docker/hooks","issue_events_url":"https://api.github.com/repos/docker/docker/issues/events{/number}","events_url":"https://api.github.com/repos/docker/docker/events","assignees_url":"https://api.github.com/repos/docker/docker/assignees{/user}","branches_url":"https://api.github.com/repos/docker/docker/branches{/branch}","tags_url":"https://api.github.com/repos/docker/docker/tags","blobs_url":"https://api.github.com/repos/docker/docker/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/docker/docker/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/docker/docker/git/refs{/sha}","trees_url":"https://api.github.com/repos/docker/docker/git/trees{/sha}","statuses_url":"https://api.github.com/repos/docker/docker/statuses/{sha}","languages_url":"https://api.github.com/repos/docker/docker/languages","stargazers_url":"https://api.github.com/repos/docker/docker/stargazers","contributors_url":"https://api.github.com/repos/docker/docker/contributors","subscribers_url":"https://api.github.com/repos/docker/docker/subscribers","subscription_url":"https://api.github.com/repos/docker/docker/subscription","commits_url":"https://api.github.com/repos/docker/docker/commits{/sha}","git_commits_url":"https://api.github.com/repos/docker/docker/git/commits{/sha}","comments_url":"https://api.github.com/repos/docker/docker/comments{/number}","issue_comment_url":"https://api.github.com/repos/docker/docker/issues/comments{/number}","contents_url":"https://api.github.com/repos/docker/docker/contents/{+path}","compare_url":"https://api.github.com/repos/docker/docker/compare/{base}...{head}","merges_url":"https://api.github.com/repos/docker/docker/merges","archive_url":"https://api.github.com/repos/docker/docker/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/docker/docker/downloads","issues_url":"https://api.github.com/repos/docker/docker/issues{/number}","pulls_url":"https://api.github.com/repos/docker/docker/pulls{/number}","milestones_url":"https://api.github.com/repos/docker/docker/milestones{/number}","notifications_url":"https://api.github.com/repos/docker/docker/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/docker/docker/labels{/name}","releases_url":"https://api.github.com/repos/docker/docker/releases{/id}","created_at":"2013-01-18T18:10:57Z","updated_at":"2016-01-16T14:32:07Z","pushed_at":"2016-01-16T17:16:55Z","git_url":"git://github.com/docker/docker.git","ssh_url":"git@github.com:docker/docker.git","clone_url":"https://github.com/docker/docker.git","svn_url":"https://github.com/docker/docker","homepage":"http://www.docker.com","size":123681,"stargazers_count":27839,"watchers_count":27839,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":7689,"mirror_url":null,"open_issues_count":1254,"forks":7689,"open_issues":1254,"watchers":27839,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/docker/docker/pulls/19389"},"html":{"href":"https://github.com/docker/docker/pull/19389"},"issue":{"href":"https://api.github.com/repos/docker/docker/issues/19389"},"comments":{"href":"https://api.github.com/repos/docker/docker/issues/19389/comments"},"review_comments":{"href":"https://api.github.com/repos/docker/docker/pulls/19389/comments"},"review_comment":{"href":"https://api.github.com/repos/docker/docker/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/docker/docker/pulls/19389/commits"},"statuses":{"href":"https://api.github.com/repos/docker/docker/statuses/11409636dc4d18f2f018d7ac85a5f76c979e78e7"}},"merged":false,"mergeable":null,"mergeable_state":"unknown","merged_by":null,"comments":2,"review_comments":0,"commits":1,"additions":1175,"deletions":5,"changed_files":14}}