{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/12797","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/12797/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/12797/comments","events_url":"https://api.github.com/repos/apache/spark/issues/12797/events","html_url":"https://github.com/apache/spark/pull/12797","id":152000278,"number":12797,"title":"[SPARK-15022][SPARK-15023][SQL][Streaming] Add support for testing against the `ProcessingTime(intervalMS > 0)` trigger and `ManualClock`","user":{"login":"lw-lin","id":15843379,"avatar_url":"https://avatars.githubusercontent.com/u/15843379?v=3","gravatar_id":"","url":"https://api.github.com/users/lw-lin","html_url":"https://github.com/lw-lin","followers_url":"https://api.github.com/users/lw-lin/followers","following_url":"https://api.github.com/users/lw-lin/following{/other_user}","gists_url":"https://api.github.com/users/lw-lin/gists{/gist_id}","starred_url":"https://api.github.com/users/lw-lin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lw-lin/subscriptions","organizations_url":"https://api.github.com/users/lw-lin/orgs","repos_url":"https://api.github.com/users/lw-lin/repos","events_url":"https://api.github.com/users/lw-lin/events{/privacy}","received_events_url":"https://api.github.com/users/lw-lin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2016-04-30T02:02:52Z","updated_at":"2016-05-03T23:52:35Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/12797","html_url":"https://github.com/apache/spark/pull/12797","diff_url":"https://github.com/apache/spark/pull/12797.diff","patch_url":"https://github.com/apache/spark/pull/12797.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nCurrently in `StreamTest`, we have a `StartStream` which will start a streaming query against trigger `ProcessTime(intervalMS = 0)` and `SystemClock`.\r\n\r\nWe also need to test cases against `ProcessTime(intervalMS > 0)`, which often requires `ManualClock`.\r\n\r\nThis patch:\r\n- fixes an issue of `ProcessingTimeExecutor`, where for a batch it should run `batchRunner` only once but might run multiple times under certain conditions;\r\n- adds support for testing against the `ProcessingTime(intervalMS > 0)` trigger and `AdvanceManualClock`, by specifying them as fields for `StartStream`, and by adding an `AdvanceClock` action;\r\n- adds a test, which takes advantage of the new `StartStream` and `AdvanceManualClock`, to test against [PR#[SPARK-14942] Reduce delay between batch construction and execution ](https://github.com/apache/spark/pull/12725).\r\n\r\n## How was this patch tested?\r\n\r\nN/A"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/216701369","html_url":"https://github.com/apache/spark/pull/12797#issuecomment-216701369","issue_url":"https://api.github.com/repos/apache/spark/issues/12797","id":216701369,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-05-03T23:52:35Z","updated_at":"2016-05-03T23:52:35Z","body":"**[Test build #57691 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57691/consoleFull)** for PR 12797 at commit [`bc89962`](https://github.com/apache/spark/commit/bc899628fa67acfbe6bb3d8e2c0ba2aefc2422a6)."}}