{"action":"created","issue":{"url":"https://api.github.com/repos/edx/edx-analytics-pipeline/issues/287","repository_url":"https://api.github.com/repos/edx/edx-analytics-pipeline","labels_url":"https://api.github.com/repos/edx/edx-analytics-pipeline/issues/287/labels{/name}","comments_url":"https://api.github.com/repos/edx/edx-analytics-pipeline/issues/287/comments","events_url":"https://api.github.com/repos/edx/edx-analytics-pipeline/issues/287/events","html_url":"https://github.com/edx/edx-analytics-pipeline/pull/287","id":174138744,"number":287,"title":" Incremental enrollment.","user":{"login":"HassanJaveed84","id":5018044,"avatar_url":"https://avatars.githubusercontent.com/u/5018044?v=3","gravatar_id":"","url":"https://api.github.com/users/HassanJaveed84","html_url":"https://github.com/HassanJaveed84","followers_url":"https://api.github.com/users/HassanJaveed84/followers","following_url":"https://api.github.com/users/HassanJaveed84/following{/other_user}","gists_url":"https://api.github.com/users/HassanJaveed84/gists{/gist_id}","starred_url":"https://api.github.com/users/HassanJaveed84/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HassanJaveed84/subscriptions","organizations_url":"https://api.github.com/users/HassanJaveed84/orgs","repos_url":"https://api.github.com/users/HassanJaveed84/repos","events_url":"https://api.github.com/users/HassanJaveed84/events{/privacy}","received_events_url":"https://api.github.com/users/HassanJaveed84/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":17,"created_at":"2016-08-30T22:10:54Z","updated_at":"2016-09-28T18:16:40Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/edx/edx-analytics-pipeline/pulls/287","html_url":"https://github.com/edx/edx-analytics-pipeline/pull/287","diff_url":"https://github.com/edx/edx-analytics-pipeline/pull/287.diff","patch_url":"https://github.com/edx/edx-analytics-pipeline/pull/287.patch"},"body":"@mulby @brianhw \r\nI started with a different approach where I used Hive partition on date to store daily enrollment. But I wasn't sure how to run a mapreduce over a hive table output(all partitions). \r\n\r\nCurrently, I just store daily enrollment on a warehouse path and another mapreduce job generates the enrollment like we do now. It's incremental in sense that we would have a daily task that just processes one days of event logs, and another that would use all the incremental task outputs and produces what we have now.\r\n\r\nIs there a way I can put daily enrollment into hive partitions and query across all the partitions ?"},"comment":{"url":"https://api.github.com/repos/edx/edx-analytics-pipeline/issues/comments/250252793","html_url":"https://github.com/edx/edx-analytics-pipeline/pull/287#issuecomment-250252793","issue_url":"https://api.github.com/repos/edx/edx-analytics-pipeline/issues/287","id":250252793,"user":{"login":"mulby","id":627177,"avatar_url":"https://avatars.githubusercontent.com/u/627177?v=3","gravatar_id":"","url":"https://api.github.com/users/mulby","html_url":"https://github.com/mulby","followers_url":"https://api.github.com/users/mulby/followers","following_url":"https://api.github.com/users/mulby/following{/other_user}","gists_url":"https://api.github.com/users/mulby/gists{/gist_id}","starred_url":"https://api.github.com/users/mulby/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mulby/subscriptions","organizations_url":"https://api.github.com/users/mulby/orgs","repos_url":"https://api.github.com/users/mulby/repos","events_url":"https://api.github.com/users/mulby/events{/privacy}","received_events_url":"https://api.github.com/users/mulby/received_events","type":"User","site_admin":false},"created_at":"2016-09-28T18:16:40Z","updated_at":"2016-09-28T18:16:40Z","body":"Perfect, squash, merge and release when ready."}}
{"actor":{"display_login":"mulby"}}