{"action":"created","comment":{"url":"https://api.github.com/repos/twitter/scalding/pulls/comments/53497235","id":53497235,"diff_hunk":"@@ -419,9 +419,22 @@ abstract class FixedPathSource(path: String*) extends FileSource {\n   def localPaths = path.toList\n   def hdfsPaths = path.toList\n \n+  override def hdfsWritePath = stripTrailingStar(super.hdfsWritePath)\n+\n   override def toString = getClass.getName + path\n   override def hashCode = toString.hashCode\n   override def equals(that: Any): Boolean = (that != null) && (that.toString == toString)\n+\n+  /**\n+    * Strip trailing '*' from the path string.\n+    */","path":"scalding-core/src/main/scala/com/twitter/scalding/FileSource.scala","position":12,"original_position":12,"commit_id":"695ab87dfba1bfc90442de6e353586a5d84aedb5","original_commit_id":"695ab87dfba1bfc90442de6e353586a5d84aedb5","user":{"login":"johnynek","id":67958,"avatar_url":"https://avatars.githubusercontent.com/u/67958?v=3","gravatar_id":"","url":"https://api.github.com/users/johnynek","html_url":"https://github.com/johnynek","followers_url":"https://api.github.com/users/johnynek/followers","following_url":"https://api.github.com/users/johnynek/following{/other_user}","gists_url":"https://api.github.com/users/johnynek/gists{/gist_id}","starred_url":"https://api.github.com/users/johnynek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/johnynek/subscriptions","organizations_url":"https://api.github.com/users/johnynek/orgs","repos_url":"https://api.github.com/users/johnynek/repos","events_url":"https://api.github.com/users/johnynek/events{/privacy}","received_events_url":"https://api.github.com/users/johnynek/received_events","type":"User","site_admin":false},"body":"can you add a link to the `TimePathedSource` method `writePathFor`?\r\n\r\nIt seems like that code is dropping the trailing `/`. Should we not do the same thing here? I think that was done to accomodate local mode which writes to one file. Hadoop seems to create a directory in any case.\r\n\r\nAlso, can we make that code use the same style `.dropRight(2)` in that case?","created_at":"2016-02-19T18:24:24Z","updated_at":"2016-02-19T18:24:24Z","html_url":"https://github.com/twitter/scalding/pull/1520#discussion_r53497235","pull_request_url":"https://api.github.com/repos/twitter/scalding/pulls/1520","_links":{"self":{"href":"https://api.github.com/repos/twitter/scalding/pulls/comments/53497235"},"html":{"href":"https://github.com/twitter/scalding/pull/1520#discussion_r53497235"},"pull_request":{"href":"https://api.github.com/repos/twitter/scalding/pulls/1520"}}},"pull_request":{"url":"https://api.github.com/repos/twitter/scalding/pulls/1520","id":59890859,"html_url":"https://github.com/twitter/scalding/pull/1520","diff_url":"https://github.com/twitter/scalding/pull/1520.diff","patch_url":"https://github.com/twitter/scalding/pull/1520.patch","issue_url":"https://api.github.com/repos/twitter/scalding/issues/1520","number":1520,"state":"open","locked":false,"title":"Update FixedPathSource to strip out '*' in paths ending with '/*' for writes","user":{"login":"piyushnarang","id":9885908,"avatar_url":"https://avatars.githubusercontent.com/u/9885908?v=3","gravatar_id":"","url":"https://api.github.com/users/piyushnarang","html_url":"https://github.com/piyushnarang","followers_url":"https://api.github.com/users/piyushnarang/followers","following_url":"https://api.github.com/users/piyushnarang/following{/other_user}","gists_url":"https://api.github.com/users/piyushnarang/gists{/gist_id}","starred_url":"https://api.github.com/users/piyushnarang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/piyushnarang/subscriptions","organizations_url":"https://api.github.com/users/piyushnarang/orgs","repos_url":"https://api.github.com/users/piyushnarang/repos","events_url":"https://api.github.com/users/piyushnarang/events{/privacy}","received_events_url":"https://api.github.com/users/piyushnarang/received_events","type":"User","site_admin":false},"body":"Currently the paths we seem to pass in for reads end up terminating in a \"/\\*\". When we end up using the same for our write paths, data is written out under a subdirectory (for example: testdata/\\*/part-0000.lzo instead of testdata/part-0000.lzo). \r\n\r\nUpdated the FixedPathSource to behave similarly to the TimePathedSource and trim the trailing '*' in these cases. ","created_at":"2016-02-19T01:14:53Z","updated_at":"2016-02-19T18:24:24Z","closed_at":null,"merged_at":null,"merge_commit_sha":"c0bf8a079c323368bd89a1bb268a9202638f42f9","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/twitter/scalding/pulls/1520/commits","review_comments_url":"https://api.github.com/repos/twitter/scalding/pulls/1520/comments","review_comment_url":"https://api.github.com/repos/twitter/scalding/pulls/comments{/number}","comments_url":"https://api.github.com/repos/twitter/scalding/issues/1520/comments","statuses_url":"https://api.github.com/repos/twitter/scalding/statuses/695ab87dfba1bfc90442de6e353586a5d84aedb5","head":{"label":"piyushnarang:develop","ref":"develop","sha":"695ab87dfba1bfc90442de6e353586a5d84aedb5","user":{"login":"piyushnarang","id":9885908,"avatar_url":"https://avatars.githubusercontent.com/u/9885908?v=3","gravatar_id":"","url":"https://api.github.com/users/piyushnarang","html_url":"https://github.com/piyushnarang","followers_url":"https://api.github.com/users/piyushnarang/followers","following_url":"https://api.github.com/users/piyushnarang/following{/other_user}","gists_url":"https://api.github.com/users/piyushnarang/gists{/gist_id}","starred_url":"https://api.github.com/users/piyushnarang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/piyushnarang/subscriptions","organizations_url":"https://api.github.com/users/piyushnarang/orgs","repos_url":"https://api.github.com/users/piyushnarang/repos","events_url":"https://api.github.com/users/piyushnarang/events{/privacy}","received_events_url":"https://api.github.com/users/piyushnarang/received_events","type":"User","site_admin":false},"repo":{"id":51805655,"name":"scalding","full_name":"piyushnarang/scalding","owner":{"login":"piyushnarang","id":9885908,"avatar_url":"https://avatars.githubusercontent.com/u/9885908?v=3","gravatar_id":"","url":"https://api.github.com/users/piyushnarang","html_url":"https://github.com/piyushnarang","followers_url":"https://api.github.com/users/piyushnarang/followers","following_url":"https://api.github.com/users/piyushnarang/following{/other_user}","gists_url":"https://api.github.com/users/piyushnarang/gists{/gist_id}","starred_url":"https://api.github.com/users/piyushnarang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/piyushnarang/subscriptions","organizations_url":"https://api.github.com/users/piyushnarang/orgs","repos_url":"https://api.github.com/users/piyushnarang/repos","events_url":"https://api.github.com/users/piyushnarang/events{/privacy}","received_events_url":"https://api.github.com/users/piyushnarang/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/piyushnarang/scalding","description":"A Scala API for Cascading","fork":true,"url":"https://api.github.com/repos/piyushnarang/scalding","forks_url":"https://api.github.com/repos/piyushnarang/scalding/forks","keys_url":"https://api.github.com/repos/piyushnarang/scalding/keys{/key_id}","collaborators_url":"https://api.github.com/repos/piyushnarang/scalding/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/piyushnarang/scalding/teams","hooks_url":"https://api.github.com/repos/piyushnarang/scalding/hooks","issue_events_url":"https://api.github.com/repos/piyushnarang/scalding/issues/events{/number}","events_url":"https://api.github.com/repos/piyushnarang/scalding/events","assignees_url":"https://api.github.com/repos/piyushnarang/scalding/assignees{/user}","branches_url":"https://api.github.com/repos/piyushnarang/scalding/branches{/branch}","tags_url":"https://api.github.com/repos/piyushnarang/scalding/tags","blobs_url":"https://api.github.com/repos/piyushnarang/scalding/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/piyushnarang/scalding/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/piyushnarang/scalding/git/refs{/sha}","trees_url":"https://api.github.com/repos/piyushnarang/scalding/git/trees{/sha}","statuses_url":"https://api.github.com/repos/piyushnarang/scalding/statuses/{sha}","languages_url":"https://api.github.com/repos/piyushnarang/scalding/languages","stargazers_url":"https://api.github.com/repos/piyushnarang/scalding/stargazers","contributors_url":"https://api.github.com/repos/piyushnarang/scalding/contributors","subscribers_url":"https://api.github.com/repos/piyushnarang/scalding/subscribers","subscription_url":"https://api.github.com/repos/piyushnarang/scalding/subscription","commits_url":"https://api.github.com/repos/piyushnarang/scalding/commits{/sha}","git_commits_url":"https://api.github.com/repos/piyushnarang/scalding/git/commits{/sha}","comments_url":"https://api.github.com/repos/piyushnarang/scalding/comments{/number}","issue_comment_url":"https://api.github.com/repos/piyushnarang/scalding/issues/comments{/number}","contents_url":"https://api.github.com/repos/piyushnarang/scalding/contents/{+path}","compare_url":"https://api.github.com/repos/piyushnarang/scalding/compare/{base}...{head}","merges_url":"https://api.github.com/repos/piyushnarang/scalding/merges","archive_url":"https://api.github.com/repos/piyushnarang/scalding/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/piyushnarang/scalding/downloads","issues_url":"https://api.github.com/repos/piyushnarang/scalding/issues{/number}","pulls_url":"https://api.github.com/repos/piyushnarang/scalding/pulls{/number}","milestones_url":"https://api.github.com/repos/piyushnarang/scalding/milestones{/number}","notifications_url":"https://api.github.com/repos/piyushnarang/scalding/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/piyushnarang/scalding/labels{/name}","releases_url":"https://api.github.com/repos/piyushnarang/scalding/releases{/id}","deployments_url":"https://api.github.com/repos/piyushnarang/scalding/deployments","created_at":"2016-02-16T03:39:09Z","updated_at":"2016-02-16T03:39:12Z","pushed_at":"2016-02-19T00:02:26Z","git_url":"git://github.com/piyushnarang/scalding.git","ssh_url":"git@github.com:piyushnarang/scalding.git","clone_url":"https://github.com/piyushnarang/scalding.git","svn_url":"https://github.com/piyushnarang/scalding","homepage":"http://twitter.com/scalding","size":13535,"stargazers_count":0,"watchers_count":0,"language":"Scala","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"develop"}},"base":{"label":"twitter:develop","ref":"develop","sha":"d22990bfeb3987bfce01172284c344354fa500f7","user":{"login":"twitter","id":50278,"avatar_url":"https://avatars.githubusercontent.com/u/50278?v=3","gravatar_id":"","url":"https://api.github.com/users/twitter","html_url":"https://github.com/twitter","followers_url":"https://api.github.com/users/twitter/followers","following_url":"https://api.github.com/users/twitter/following{/other_user}","gists_url":"https://api.github.com/users/twitter/gists{/gist_id}","starred_url":"https://api.github.com/users/twitter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twitter/subscriptions","organizations_url":"https://api.github.com/users/twitter/orgs","repos_url":"https://api.github.com/users/twitter/repos","events_url":"https://api.github.com/users/twitter/events{/privacy}","received_events_url":"https://api.github.com/users/twitter/received_events","type":"Organization","site_admin":false},"repo":{"id":3146558,"name":"scalding","full_name":"twitter/scalding","owner":{"login":"twitter","id":50278,"avatar_url":"https://avatars.githubusercontent.com/u/50278?v=3","gravatar_id":"","url":"https://api.github.com/users/twitter","html_url":"https://github.com/twitter","followers_url":"https://api.github.com/users/twitter/followers","following_url":"https://api.github.com/users/twitter/following{/other_user}","gists_url":"https://api.github.com/users/twitter/gists{/gist_id}","starred_url":"https://api.github.com/users/twitter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twitter/subscriptions","organizations_url":"https://api.github.com/users/twitter/orgs","repos_url":"https://api.github.com/users/twitter/repos","events_url":"https://api.github.com/users/twitter/events{/privacy}","received_events_url":"https://api.github.com/users/twitter/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/twitter/scalding","description":"A Scala API for Cascading","fork":false,"url":"https://api.github.com/repos/twitter/scalding","forks_url":"https://api.github.com/repos/twitter/scalding/forks","keys_url":"https://api.github.com/repos/twitter/scalding/keys{/key_id}","collaborators_url":"https://api.github.com/repos/twitter/scalding/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/twitter/scalding/teams","hooks_url":"https://api.github.com/repos/twitter/scalding/hooks","issue_events_url":"https://api.github.com/repos/twitter/scalding/issues/events{/number}","events_url":"https://api.github.com/repos/twitter/scalding/events","assignees_url":"https://api.github.com/repos/twitter/scalding/assignees{/user}","branches_url":"https://api.github.com/repos/twitter/scalding/branches{/branch}","tags_url":"https://api.github.com/repos/twitter/scalding/tags","blobs_url":"https://api.github.com/repos/twitter/scalding/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/twitter/scalding/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/twitter/scalding/git/refs{/sha}","trees_url":"https://api.github.com/repos/twitter/scalding/git/trees{/sha}","statuses_url":"https://api.github.com/repos/twitter/scalding/statuses/{sha}","languages_url":"https://api.github.com/repos/twitter/scalding/languages","stargazers_url":"https://api.github.com/repos/twitter/scalding/stargazers","contributors_url":"https://api.github.com/repos/twitter/scalding/contributors","subscribers_url":"https://api.github.com/repos/twitter/scalding/subscribers","subscription_url":"https://api.github.com/repos/twitter/scalding/subscription","commits_url":"https://api.github.com/repos/twitter/scalding/commits{/sha}","git_commits_url":"https://api.github.com/repos/twitter/scalding/git/commits{/sha}","comments_url":"https://api.github.com/repos/twitter/scalding/comments{/number}","issue_comment_url":"https://api.github.com/repos/twitter/scalding/issues/comments{/number}","contents_url":"https://api.github.com/repos/twitter/scalding/contents/{+path}","compare_url":"https://api.github.com/repos/twitter/scalding/compare/{base}...{head}","merges_url":"https://api.github.com/repos/twitter/scalding/merges","archive_url":"https://api.github.com/repos/twitter/scalding/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/twitter/scalding/downloads","issues_url":"https://api.github.com/repos/twitter/scalding/issues{/number}","pulls_url":"https://api.github.com/repos/twitter/scalding/pulls{/number}","milestones_url":"https://api.github.com/repos/twitter/scalding/milestones{/number}","notifications_url":"https://api.github.com/repos/twitter/scalding/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/twitter/scalding/labels{/name}","releases_url":"https://api.github.com/repos/twitter/scalding/releases{/id}","deployments_url":"https://api.github.com/repos/twitter/scalding/deployments","created_at":"2012-01-10T16:22:08Z","updated_at":"2016-02-19T12:37:14Z","pushed_at":"2016-02-19T05:08:33Z","git_url":"git://github.com/twitter/scalding.git","ssh_url":"git@github.com:twitter/scalding.git","clone_url":"https://github.com/twitter/scalding.git","svn_url":"https://github.com/twitter/scalding","homepage":"http://twitter.com/scalding","size":15162,"stargazers_count":2437,"watchers_count":2437,"language":"Scala","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":568,"mirror_url":null,"open_issues_count":260,"forks":568,"open_issues":260,"watchers":2437,"default_branch":"develop"}},"_links":{"self":{"href":"https://api.github.com/repos/twitter/scalding/pulls/1520"},"html":{"href":"https://github.com/twitter/scalding/pull/1520"},"issue":{"href":"https://api.github.com/repos/twitter/scalding/issues/1520"},"comments":{"href":"https://api.github.com/repos/twitter/scalding/issues/1520/comments"},"review_comments":{"href":"https://api.github.com/repos/twitter/scalding/pulls/1520/comments"},"review_comment":{"href":"https://api.github.com/repos/twitter/scalding/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/twitter/scalding/pulls/1520/commits"},"statuses":{"href":"https://api.github.com/repos/twitter/scalding/statuses/695ab87dfba1bfc90442de6e353586a5d84aedb5"}}}}