{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/12616","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/12616/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/12616/comments","events_url":"https://api.github.com/repos/apache/spark/issues/12616/events","html_url":"https://github.com/apache/spark/pull/12616","id":150444014,"number":12616,"title":"[SPARK-14837][SQL][STREAMING] Added support in file stream source for reading new files added to subdirs","user":{"login":"tdas","id":663212,"avatar_url":"https://avatars.githubusercontent.com/u/663212?v=3","gravatar_id":"","url":"https://api.github.com/users/tdas","html_url":"https://github.com/tdas","followers_url":"https://api.github.com/users/tdas/followers","following_url":"https://api.github.com/users/tdas/following{/other_user}","gists_url":"https://api.github.com/users/tdas/gists{/gist_id}","starred_url":"https://api.github.com/users/tdas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tdas/subscriptions","organizations_url":"https://api.github.com/users/tdas/orgs","repos_url":"https://api.github.com/users/tdas/repos","events_url":"https://api.github.com/users/tdas/events{/privacy}","received_events_url":"https://api.github.com/users/tdas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":17,"created_at":"2016-04-22T18:44:14Z","updated_at":"2016-05-10T23:41:04Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/12616","html_url":"https://github.com/apache/spark/pull/12616","diff_url":"https://github.com/apache/spark/pull/12616.diff","patch_url":"https://github.com/apache/spark/pull/12616.patch"},"body":"## What changes were proposed in this pull request?\r\nCurrently, file stream source can only find new files if they appear in the directory given to the source, but not if they appear in subdirs. This PR add support for providing glob patterns when creating file stream source so that it can find new files in nested directories based on the glob pattern.\r\n\r\n## How was this patch tested?\r\n\r\nUnit test that tests when new files are discovered with globs and partitioned directories. "},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/218323829","html_url":"https://github.com/apache/spark/pull/12616#issuecomment-218323829","issue_url":"https://api.github.com/repos/apache/spark/issues/12616","id":218323829,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2016-05-10T23:41:04Z","updated_at":"2016-05-10T23:41:04Z","body":"Merged build finished. Test PASSed."}}