{"action":"created","issue":{"url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/issues/32","repository_url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems","labels_url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/issues/32/labels{/name}","comments_url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/issues/32/comments","events_url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/issues/32/events","html_url":"https://github.com/anhnongdan/Spark1.6_Problems/issues/32","id":331040005,"node_id":"MDU6SXNzdWUzMzEwNDAwMDU=","number":32,"title":"[Practice] Improve performance of daily IMEI calculation","user":{"login":"anhnongdan","id":9796214,"node_id":"MDQ6VXNlcjk3OTYyMTQ=","avatar_url":"https://avatars3.githubusercontent.com/u/9796214?v=4","gravatar_id":"","url":"https://api.github.com/users/anhnongdan","html_url":"https://github.com/anhnongdan","followers_url":"https://api.github.com/users/anhnongdan/followers","following_url":"https://api.github.com/users/anhnongdan/following{/other_user}","gists_url":"https://api.github.com/users/anhnongdan/gists{/gist_id}","starred_url":"https://api.github.com/users/anhnongdan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anhnongdan/subscriptions","organizations_url":"https://api.github.com/users/anhnongdan/orgs","repos_url":"https://api.github.com/users/anhnongdan/repos","events_url":"https://api.github.com/users/anhnongdan/events{/privacy}","received_events_url":"https://api.github.com/users/anhnongdan/received_events","type":"User","site_admin":false},"labels":[{"id":804953406,"node_id":"MDU6TGFiZWw4MDQ5NTM0MDY=","url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/labels/BenchMark","name":"BenchMark","color":"0e8a16","default":false},{"id":787788467,"node_id":"MDU6TGFiZWw3ODc3ODg0Njc=","url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/labels/Disk","name":"Disk","color":"0e8a16","default":false},{"id":787789046,"node_id":"MDU6TGFiZWw3ODc3ODkwNDY=","url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/labels/In/Out","name":"In/Out","color":"006b75","default":false},{"id":787788441,"node_id":"MDU6TGFiZWw3ODc3ODg0NDE=","url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/labels/Memory","name":"Memory","color":"b60205","default":false},{"id":788579832,"node_id":"MDU6TGFiZWw3ODg1Nzk4MzI=","url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/labels/Performance","name":"Performance","color":"1d76db","default":false},{"id":788120268,"node_id":"MDU6TGFiZWw3ODgxMjAyNjg=","url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/labels/Platform","name":"Platform","color":"54f98b","default":false},{"id":788698399,"node_id":"MDU6TGFiZWw3ODg2OTgzOTk=","url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/labels/best-practice","name":"best-practice","color":"d93f0b","default":false},{"id":787787569,"node_id":"MDU6TGFiZWw3ODc3ODc1Njk=","url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2018-06-11T04:13:52Z","updated_at":"2018-06-11T10:10:32Z","closed_at":null,"author_association":"OWNER","body":"https://10.8.0.1:8806/notebooks/thangnguyen/04_Raw_Features/IMEI_vs_TAC.ipynb\r\n\r\nThe calculation is pretty heavy when calculating on full population. \r\nNeed to extract people with multiple IMEIs used a day and rank over [num_calls, duration] and [call min time] to get top IMEI and latest IMEI.\r\n\r\nOn VT tc_histories of 1 day is around 400M records."},"comment":{"url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/issues/comments/396192705","html_url":"https://github.com/anhnongdan/Spark1.6_Problems/issues/32#issuecomment-396192705","issue_url":"https://api.github.com/repos/anhnongdan/Spark1.6_Problems/issues/32","id":396192705,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NjE5MjcwNQ==","user":{"login":"anhnongdan","id":9796214,"node_id":"MDQ6VXNlcjk3OTYyMTQ=","avatar_url":"https://avatars3.githubusercontent.com/u/9796214?v=4","gravatar_id":"","url":"https://api.github.com/users/anhnongdan","html_url":"https://github.com/anhnongdan","followers_url":"https://api.github.com/users/anhnongdan/followers","following_url":"https://api.github.com/users/anhnongdan/following{/other_user}","gists_url":"https://api.github.com/users/anhnongdan/gists{/gist_id}","starred_url":"https://api.github.com/users/anhnongdan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anhnongdan/subscriptions","organizations_url":"https://api.github.com/users/anhnongdan/orgs","repos_url":"https://api.github.com/users/anhnongdan/repos","events_url":"https://api.github.com/users/anhnongdan/events{/privacy}","received_events_url":"https://api.github.com/users/anhnongdan/received_events","type":"User","site_admin":false},"created_at":"2018-06-11T10:10:32Z","updated_at":"2018-06-11T10:10:32Z","author_association":"OWNER","body":"I eliminate all cache() at intermediate df (single_imei_usage, multi_imei_usage, etc..).\r\nFirst, parallel computing happened.\r\n\r\n![screen shot 2018-06-11 at 5 07 27 pm](https://user-images.githubusercontent.com/9796214/41225980-533cd0c6-6d9a-11e8-919f-6f1d12daef0f.png)\r\n"}}
{"actor":{"display_login":"anhnongdan"}}