{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/21721","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/21721/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/21721/comments","events_url":"https://api.github.com/repos/apache/spark/issues/21721/events","html_url":"https://github.com/apache/spark/pull/21721","id":338780252,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk5NjI3NDkw","number":21721,"title":"[SPARK-24748][SS] Support for reporting custom metrics via StreamingQuery Progress","user":{"login":"arunmahadevan","id":6792890,"node_id":"MDQ6VXNlcjY3OTI4OTA=","avatar_url":"https://avatars0.githubusercontent.com/u/6792890?v=4","gravatar_id":"","url":"https://api.github.com/users/arunmahadevan","html_url":"https://github.com/arunmahadevan","followers_url":"https://api.github.com/users/arunmahadevan/followers","following_url":"https://api.github.com/users/arunmahadevan/following{/other_user}","gists_url":"https://api.github.com/users/arunmahadevan/gists{/gist_id}","starred_url":"https://api.github.com/users/arunmahadevan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arunmahadevan/subscriptions","organizations_url":"https://api.github.com/users/arunmahadevan/orgs","repos_url":"https://api.github.com/users/arunmahadevan/repos","events_url":"https://api.github.com/users/arunmahadevan/events{/privacy}","received_events_url":"https://api.github.com/users/arunmahadevan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":10,"created_at":"2018-07-06T02:04:46Z","updated_at":"2018-07-11T18:10:08Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/21721","html_url":"https://github.com/apache/spark/pull/21721","diff_url":"https://github.com/apache/spark/pull/21721.diff","patch_url":"https://github.com/apache/spark/pull/21721.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nCurrently the Structured Streaming sources and sinks does not have a way to report custom metrics. Providing an option to report custom metrics and making it available via Streaming Query progress can enable sources and sinks to report custom progress information (E.g. the lag metrics for Kafka source).\r\n\r\nSimilar metrics can be reported for Sinks as well, but would like to get initial feedback before proceeding further.\r\n\r\n## How was this patch tested?\r\n\r\nNew and existing unit tests.\r\n\r\nPlease review http://spark.apache.org/contributing.html before opening a pull request.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/404261572","html_url":"https://github.com/apache/spark/pull/21721#issuecomment-404261572","issue_url":"https://api.github.com/repos/apache/spark/issues/21721","id":404261572,"node_id":"MDEyOklzc3VlQ29tbWVudDQwNDI2MTU3Mg==","user":{"login":"AmplabJenkins","id":4031291,"node_id":"MDQ6VXNlcjQwMzEyOTE=","avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2018-07-11T18:10:08Z","updated_at":"2018-07-11T18:10:08Z","author_association":"NONE","body":"Test FAILed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92870/\nTest FAILed."}}
{"actor":{"display_login":"AmplabJenkins"}}