{"action":"created","comment":{"url":"https://api.github.com/repos/yahoo/fili/pulls/comments/78077802","id":78077802,"diff_hunk":"@@ -561,6 +563,102 @@ There are, however, a few catches to this:\n - Sorting is only applied at the Druid level. Therefore, the results of a sort are not guaranteed to be accurate if Fili \n performs any post-Druid calculations on one of the metrics that you are sorting on.\n \n+Asynchronous Queries\n+--------------------\n+Fili supports asynchronous data queries. A new parameter `asyncAfter` is added on data queries. The `asyncAfter`\n+parameter will control whether a data query should always be synchronous, or transition from synchronous to asynchronous\n+ on the fly. If `asyncAfter=never` then Fili will wait indefinitely for the data, and hold the connection with the\n+ client open as long as allowed by the network. This will be the default. However, the default behavior of `asyncAfter`\n+ may be modified by setting the `default_asyncAfter` configuration parameter. If `asyncAfter=0`, the query is\n+ asynchronous immediately.\n+\n+If it takes less time than the timeout for the data to be sent back, then the results are sent to the client.\n+If `asyncAfter=t` for `t` an integer in milliseconds then the query will start synchronously. If the query takes longer\n+than `t` milliseconds, then the query becomes asynchronous. If the timeout passes, and the data has not come back, then\n+the user receives a `202 Accepted` response and the [job meta-data](#job-meta-data).\n+\n+### Jobs Endpoint\n+The jobs endpoint is the one stop shop for queries about asynchronous jobs. This endpoint is responsible for:\n+\n+1. Provide a list of [all jobs](#get-ting-summary-of-all-jobs) in the system.\n+2. Providing the status of a [particular job](#get-ting-job-status) queried via the `jobs/TICKET` resource.\n+3. Providing access to the [results](#get-ting-results) via the `jobs/TICKET/results` resource.\n+\n+#### GET-ting summary of all jobs\n+A user may get the status of all jobs by sending a `GET` to `jobs` endpoint.\n+\n+```\n+https://HOST:PORT/v1/jobs\n+```\n+If no jobs are available in the system, an empty collection is returned.\n+\n+The `jobs` endpoint supports filtering on job fields (i.e. `userId`, `status`), using the same syntax as the\n+[data endpoint filters](#filtering). For example:\n+\n+userId-eq[greg, joan], status-eq[success]\n+\n+resolves into the following boolean operation:\n+\n+(userId = greg OR userId = joan) AND status = success\n+\n+which will return only those Jobs created by `greg` and `joan` that have completed successfully.\n+\n+#### GET-ting Job Status\n+When the user sends a `GET` request to `jobs/TICKET`, Fili will look up the specified ticket and return the job's\n+meta-data as follows:\n+\n+###### Job meta-data\n+```json\n+{\n+    \"query\": \"https://HOST:PORT/v1/data/QUERY\",\n+    \"results\": \"https://HOST:PORT/v1/jobs/TICKET/results\",\n+    \"syncResults\": \"https://HOST:PORT/v1/jobs/TICKET/results?asyncAfter=never\",\n+    \"self\": \"https://HOST:PORT/v1/jobs/TICKET\",\n+    \"status\": ONE OF [\"pending\", \"success\", \"error\"],\n+    \"jobTicket\": \"TICKET\",\n+    \"dateCreated\": \"ISO 8601 DATETIME\",\n+    \"dateUpdated\": \"ISO 8601 DATETIME\",\n+    \"userId\": \"Foo\"\n+}\n+```\n+* `query` is the original query\n+* `results` provides a link to the data, whether it is fully synchronous or switches from\n+   synchronous to asynchronous after a timeout depends on the default setting of `asyncAfter`.\n+* `syncResults` provides a synchronous link to the data (note the `asyncAfter=never` parameter)\n+* `self` provides a link that returns the most up-to-date version of this job\n+* `status` indicates the status of the results pointed to by this ticket\n+    - `pending` - The job is being worked on\n+    - `success` - The job has been completed successfully\n+    - `error` - The job failed with an error\n+    - `canceled` - The job was canceled by the user (coming soon)\n+* `jobTicket` is a unique identifier for the job\n+* `dateCreated` is the date on which the job was created\n+* `dateUpdated` when the job's status was last updated\n+* `userId` is an identifier for the user who submitted this job\n+\n+If the ticket is not available in the system, we get a 404 error with the message `No job found with job ticket TICKET`\n+\n+#### GET-ting Results\n+The user may access the results of a query by sending a `GET` request to `jobs/TICKET/results`. This\n+resource takes the following parameters:\n+\n+ 1. **`format`** - Allows the user to specify a response format, i.e. csv, or JSON. This behaves\n+just like the [`format`](#response-format) parameter on queries sent to the `data` endpoint.\n+\n+2. **`page`, `perPage`** - The [pagination](#pagination--limit) parameters. Their behavior is the same as when sending\n+a query to the `data` endpoint, and allow the user to get pages of the results.","path":"api-docs.md","position":96,"original_position":96,"commit_id":"b38909551c26f5c6992c33756e478dc427b2480c","original_commit_id":"b38909551c26f5c6992c33756e478dc427b2480c","user":{"login":"michael-mclawhorn","id":4105159,"avatar_url":"https://avatars.githubusercontent.com/u/4105159?v=3","gravatar_id":"","url":"https://api.github.com/users/michael-mclawhorn","html_url":"https://github.com/michael-mclawhorn","followers_url":"https://api.github.com/users/michael-mclawhorn/followers","following_url":"https://api.github.com/users/michael-mclawhorn/following{/other_user}","gists_url":"https://api.github.com/users/michael-mclawhorn/gists{/gist_id}","starred_url":"https://api.github.com/users/michael-mclawhorn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/michael-mclawhorn/subscriptions","organizations_url":"https://api.github.com/users/michael-mclawhorn/orgs","repos_url":"https://api.github.com/users/michael-mclawhorn/repos","events_url":"https://api.github.com/users/michael-mclawhorn/events{/privacy}","received_events_url":"https://api.github.com/users/michael-mclawhorn/received_events","type":"User","site_admin":false},"body":"Should we flag this as also 'coming soon'?  It's blocked on: \r\nhttps://github.com/yahoo/fili/issues/36","created_at":"2016-09-08T19:56:20Z","updated_at":"2016-09-08T19:56:20Z","html_url":"https://github.com/yahoo/fili/pull/29#discussion_r78077802","pull_request_url":"https://api.github.com/repos/yahoo/fili/pulls/29","_links":{"self":{"href":"https://api.github.com/repos/yahoo/fili/pulls/comments/78077802"},"html":{"href":"https://github.com/yahoo/fili/pull/29#discussion_r78077802"},"pull_request":{"href":"https://api.github.com/repos/yahoo/fili/pulls/29"}}},"pull_request":{"url":"https://api.github.com/repos/yahoo/fili/pulls/29","id":83532636,"html_url":"https://github.com/yahoo/fili/pull/29","diff_url":"https://github.com/yahoo/fili/pull/29.diff","patch_url":"https://github.com/yahoo/fili/pull/29.patch","issue_url":"https://api.github.com/repos/yahoo/fili/issues/29","number":29,"state":"open","locked":false,"title":"Add jobs endpoint documentation","user":{"login":"venisa","id":5899037,"avatar_url":"https://avatars.githubusercontent.com/u/5899037?v=3","gravatar_id":"","url":"https://api.github.com/users/venisa","html_url":"https://github.com/venisa","followers_url":"https://api.github.com/users/venisa/followers","following_url":"https://api.github.com/users/venisa/following{/other_user}","gists_url":"https://api.github.com/users/venisa/gists{/gist_id}","starred_url":"https://api.github.com/users/venisa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/venisa/subscriptions","organizations_url":"https://api.github.com/users/venisa/orgs","repos_url":"https://api.github.com/users/venisa/repos","events_url":"https://api.github.com/users/venisa/events{/privacy}","received_events_url":"https://api.github.com/users/venisa/received_events","type":"User","site_admin":false},"body":"Add documentation for Jobs endpoint.\r\n\r\nPlease note: \r\nI have provided a very brief description for async. However, the focus of this PR is to document the Jobs endpoint.\r\n\r\n@cdeszaq @archolewa you guys had reviewed this when this was opened in bard.\r\nI addressed all outstanding comments and added a new commit for filters on jobs endpoint","created_at":"2016-08-31T21:27:04Z","updated_at":"2016-09-08T19:56:20Z","closed_at":null,"merged_at":null,"merge_commit_sha":"94cfddddda5a4dc95c9d9eef465cb0f4c61d9314","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/yahoo/fili/pulls/29/commits","review_comments_url":"https://api.github.com/repos/yahoo/fili/pulls/29/comments","review_comment_url":"https://api.github.com/repos/yahoo/fili/pulls/comments{/number}","comments_url":"https://api.github.com/repos/yahoo/fili/issues/29/comments","statuses_url":"https://api.github.com/repos/yahoo/fili/statuses/b38909551c26f5c6992c33756e478dc427b2480c","head":{"label":"yahoo:Add-Jobs-endpoint-documentation","ref":"Add-Jobs-endpoint-documentation","sha":"b38909551c26f5c6992c33756e478dc427b2480c","user":{"login":"yahoo","id":16574,"avatar_url":"https://avatars.githubusercontent.com/u/16574?v=3","gravatar_id":"","url":"https://api.github.com/users/yahoo","html_url":"https://github.com/yahoo","followers_url":"https://api.github.com/users/yahoo/followers","following_url":"https://api.github.com/users/yahoo/following{/other_user}","gists_url":"https://api.github.com/users/yahoo/gists{/gist_id}","starred_url":"https://api.github.com/users/yahoo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yahoo/subscriptions","organizations_url":"https://api.github.com/users/yahoo/orgs","repos_url":"https://api.github.com/users/yahoo/repos","events_url":"https://api.github.com/users/yahoo/events{/privacy}","received_events_url":"https://api.github.com/users/yahoo/received_events","type":"Organization","site_admin":false},"repo":{"id":60879636,"name":"fili","full_name":"yahoo/fili","owner":{"login":"yahoo","id":16574,"avatar_url":"https://avatars.githubusercontent.com/u/16574?v=3","gravatar_id":"","url":"https://api.github.com/users/yahoo","html_url":"https://github.com/yahoo","followers_url":"https://api.github.com/users/yahoo/followers","following_url":"https://api.github.com/users/yahoo/following{/other_user}","gists_url":"https://api.github.com/users/yahoo/gists{/gist_id}","starred_url":"https://api.github.com/users/yahoo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yahoo/subscriptions","organizations_url":"https://api.github.com/users/yahoo/orgs","repos_url":"https://api.github.com/users/yahoo/repos","events_url":"https://api.github.com/users/yahoo/events{/privacy}","received_events_url":"https://api.github.com/users/yahoo/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/yahoo/fili","description":"","fork":false,"url":"https://api.github.com/repos/yahoo/fili","forks_url":"https://api.github.com/repos/yahoo/fili/forks","keys_url":"https://api.github.com/repos/yahoo/fili/keys{/key_id}","collaborators_url":"https://api.github.com/repos/yahoo/fili/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/yahoo/fili/teams","hooks_url":"https://api.github.com/repos/yahoo/fili/hooks","issue_events_url":"https://api.github.com/repos/yahoo/fili/issues/events{/number}","events_url":"https://api.github.com/repos/yahoo/fili/events","assignees_url":"https://api.github.com/repos/yahoo/fili/assignees{/user}","branches_url":"https://api.github.com/repos/yahoo/fili/branches{/branch}","tags_url":"https://api.github.com/repos/yahoo/fili/tags","blobs_url":"https://api.github.com/repos/yahoo/fili/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/yahoo/fili/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/yahoo/fili/git/refs{/sha}","trees_url":"https://api.github.com/repos/yahoo/fili/git/trees{/sha}","statuses_url":"https://api.github.com/repos/yahoo/fili/statuses/{sha}","languages_url":"https://api.github.com/repos/yahoo/fili/languages","stargazers_url":"https://api.github.com/repos/yahoo/fili/stargazers","contributors_url":"https://api.github.com/repos/yahoo/fili/contributors","subscribers_url":"https://api.github.com/repos/yahoo/fili/subscribers","subscription_url":"https://api.github.com/repos/yahoo/fili/subscription","commits_url":"https://api.github.com/repos/yahoo/fili/commits{/sha}","git_commits_url":"https://api.github.com/repos/yahoo/fili/git/commits{/sha}","comments_url":"https://api.github.com/repos/yahoo/fili/comments{/number}","issue_comment_url":"https://api.github.com/repos/yahoo/fili/issues/comments{/number}","contents_url":"https://api.github.com/repos/yahoo/fili/contents/{+path}","compare_url":"https://api.github.com/repos/yahoo/fili/compare/{base}...{head}","merges_url":"https://api.github.com/repos/yahoo/fili/merges","archive_url":"https://api.github.com/repos/yahoo/fili/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/yahoo/fili/downloads","issues_url":"https://api.github.com/repos/yahoo/fili/issues{/number}","pulls_url":"https://api.github.com/repos/yahoo/fili/pulls{/number}","milestones_url":"https://api.github.com/repos/yahoo/fili/milestones{/number}","notifications_url":"https://api.github.com/repos/yahoo/fili/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/yahoo/fili/labels{/name}","releases_url":"https://api.github.com/repos/yahoo/fili/releases{/id}","deployments_url":"https://api.github.com/repos/yahoo/fili/deployments","created_at":"2016-06-10T22:07:59Z","updated_at":"2016-08-31T15:24:19Z","pushed_at":"2016-09-08T19:34:18Z","git_url":"git://github.com/yahoo/fili.git","ssh_url":"git@github.com:yahoo/fili.git","clone_url":"https://github.com/yahoo/fili.git","svn_url":"https://github.com/yahoo/fili","homepage":null,"size":1272,"stargazers_count":9,"watchers_count":9,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6,"mirror_url":null,"open_issues_count":15,"forks":6,"open_issues":15,"watchers":9,"default_branch":"master"}},"base":{"label":"yahoo:master","ref":"master","sha":"0c154a64602361ce43f288450975f5fb273bcc8a","user":{"login":"yahoo","id":16574,"avatar_url":"https://avatars.githubusercontent.com/u/16574?v=3","gravatar_id":"","url":"https://api.github.com/users/yahoo","html_url":"https://github.com/yahoo","followers_url":"https://api.github.com/users/yahoo/followers","following_url":"https://api.github.com/users/yahoo/following{/other_user}","gists_url":"https://api.github.com/users/yahoo/gists{/gist_id}","starred_url":"https://api.github.com/users/yahoo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yahoo/subscriptions","organizations_url":"https://api.github.com/users/yahoo/orgs","repos_url":"https://api.github.com/users/yahoo/repos","events_url":"https://api.github.com/users/yahoo/events{/privacy}","received_events_url":"https://api.github.com/users/yahoo/received_events","type":"Organization","site_admin":false},"repo":{"id":60879636,"name":"fili","full_name":"yahoo/fili","owner":{"login":"yahoo","id":16574,"avatar_url":"https://avatars.githubusercontent.com/u/16574?v=3","gravatar_id":"","url":"https://api.github.com/users/yahoo","html_url":"https://github.com/yahoo","followers_url":"https://api.github.com/users/yahoo/followers","following_url":"https://api.github.com/users/yahoo/following{/other_user}","gists_url":"https://api.github.com/users/yahoo/gists{/gist_id}","starred_url":"https://api.github.com/users/yahoo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yahoo/subscriptions","organizations_url":"https://api.github.com/users/yahoo/orgs","repos_url":"https://api.github.com/users/yahoo/repos","events_url":"https://api.github.com/users/yahoo/events{/privacy}","received_events_url":"https://api.github.com/users/yahoo/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/yahoo/fili","description":"","fork":false,"url":"https://api.github.com/repos/yahoo/fili","forks_url":"https://api.github.com/repos/yahoo/fili/forks","keys_url":"https://api.github.com/repos/yahoo/fili/keys{/key_id}","collaborators_url":"https://api.github.com/repos/yahoo/fili/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/yahoo/fili/teams","hooks_url":"https://api.github.com/repos/yahoo/fili/hooks","issue_events_url":"https://api.github.com/repos/yahoo/fili/issues/events{/number}","events_url":"https://api.github.com/repos/yahoo/fili/events","assignees_url":"https://api.github.com/repos/yahoo/fili/assignees{/user}","branches_url":"https://api.github.com/repos/yahoo/fili/branches{/branch}","tags_url":"https://api.github.com/repos/yahoo/fili/tags","blobs_url":"https://api.github.com/repos/yahoo/fili/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/yahoo/fili/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/yahoo/fili/git/refs{/sha}","trees_url":"https://api.github.com/repos/yahoo/fili/git/trees{/sha}","statuses_url":"https://api.github.com/repos/yahoo/fili/statuses/{sha}","languages_url":"https://api.github.com/repos/yahoo/fili/languages","stargazers_url":"https://api.github.com/repos/yahoo/fili/stargazers","contributors_url":"https://api.github.com/repos/yahoo/fili/contributors","subscribers_url":"https://api.github.com/repos/yahoo/fili/subscribers","subscription_url":"https://api.github.com/repos/yahoo/fili/subscription","commits_url":"https://api.github.com/repos/yahoo/fili/commits{/sha}","git_commits_url":"https://api.github.com/repos/yahoo/fili/git/commits{/sha}","comments_url":"https://api.github.com/repos/yahoo/fili/comments{/number}","issue_comment_url":"https://api.github.com/repos/yahoo/fili/issues/comments{/number}","contents_url":"https://api.github.com/repos/yahoo/fili/contents/{+path}","compare_url":"https://api.github.com/repos/yahoo/fili/compare/{base}...{head}","merges_url":"https://api.github.com/repos/yahoo/fili/merges","archive_url":"https://api.github.com/repos/yahoo/fili/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/yahoo/fili/downloads","issues_url":"https://api.github.com/repos/yahoo/fili/issues{/number}","pulls_url":"https://api.github.com/repos/yahoo/fili/pulls{/number}","milestones_url":"https://api.github.com/repos/yahoo/fili/milestones{/number}","notifications_url":"https://api.github.com/repos/yahoo/fili/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/yahoo/fili/labels{/name}","releases_url":"https://api.github.com/repos/yahoo/fili/releases{/id}","deployments_url":"https://api.github.com/repos/yahoo/fili/deployments","created_at":"2016-06-10T22:07:59Z","updated_at":"2016-08-31T15:24:19Z","pushed_at":"2016-09-08T19:34:18Z","git_url":"git://github.com/yahoo/fili.git","ssh_url":"git@github.com:yahoo/fili.git","clone_url":"https://github.com/yahoo/fili.git","svn_url":"https://github.com/yahoo/fili","homepage":null,"size":1272,"stargazers_count":9,"watchers_count":9,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":6,"mirror_url":null,"open_issues_count":15,"forks":6,"open_issues":15,"watchers":9,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/yahoo/fili/pulls/29"},"html":{"href":"https://github.com/yahoo/fili/pull/29"},"issue":{"href":"https://api.github.com/repos/yahoo/fili/issues/29"},"comments":{"href":"https://api.github.com/repos/yahoo/fili/issues/29/comments"},"review_comments":{"href":"https://api.github.com/repos/yahoo/fili/pulls/29/comments"},"review_comment":{"href":"https://api.github.com/repos/yahoo/fili/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/yahoo/fili/pulls/29/commits"},"statuses":{"href":"https://api.github.com/repos/yahoo/fili/statuses/b38909551c26f5c6992c33756e478dc427b2480c"}}}}
{"actor":{"display_login":"michael-mclawhorn"}}