{"action":"created","issue":{"url":"https://api.github.com/repos/usc-isi-i2/datamart/issues/50","repository_url":"https://api.github.com/repos/usc-isi-i2/datamart","labels_url":"https://api.github.com/repos/usc-isi-i2/datamart/issues/50/labels{/name}","comments_url":"https://api.github.com/repos/usc-isi-i2/datamart/issues/50/comments","events_url":"https://api.github.com/repos/usc-isi-i2/datamart/issues/50/events","html_url":"https://github.com/usc-isi-i2/datamart/issues/50","id":381385360,"node_id":"MDU6SXNzdWUzODEzODUzNjA=","number":50,"title":"Breakdown WorldBank data into chunks for materialization","user":{"login":"RqS","id":6547905,"node_id":"MDQ6VXNlcjY1NDc5MDU=","avatar_url":"https://avatars2.githubusercontent.com/u/6547905?v=4","gravatar_id":"","url":"https://api.github.com/users/RqS","html_url":"https://github.com/RqS","followers_url":"https://api.github.com/users/RqS/followers","following_url":"https://api.github.com/users/RqS/following{/other_user}","gists_url":"https://api.github.com/users/RqS/gists{/gist_id}","starred_url":"https://api.github.com/users/RqS/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RqS/subscriptions","organizations_url":"https://api.github.com/users/RqS/orgs","repos_url":"https://api.github.com/users/RqS/repos","events_url":"https://api.github.com/users/RqS/events{/privacy}","received_events_url":"https://api.github.com/users/RqS/received_events","type":"User","site_admin":false},"labels":[{"id":1070690997,"node_id":"MDU6TGFiZWwxMDcwNjkwOTk3","url":"https://api.github.com/repos/usc-isi-i2/datamart/labels/enhancement","name":"enhancement","color":"a2eeef","default":true},{"id":1070691008,"node_id":"MDU6TGFiZWwxMDcwNjkxMDA4","url":"https://api.github.com/repos/usc-isi-i2/datamart/labels/question","name":"question","color":"d876e3","default":true}],"state":"open","locked":false,"assignee":{"login":"szeke","id":1074599,"node_id":"MDQ6VXNlcjEwNzQ1OTk=","avatar_url":"https://avatars2.githubusercontent.com/u/1074599?v=4","gravatar_id":"","url":"https://api.github.com/users/szeke","html_url":"https://github.com/szeke","followers_url":"https://api.github.com/users/szeke/followers","following_url":"https://api.github.com/users/szeke/following{/other_user}","gists_url":"https://api.github.com/users/szeke/gists{/gist_id}","starred_url":"https://api.github.com/users/szeke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/szeke/subscriptions","organizations_url":"https://api.github.com/users/szeke/orgs","repos_url":"https://api.github.com/users/szeke/repos","events_url":"https://api.github.com/users/szeke/events{/privacy}","received_events_url":"https://api.github.com/users/szeke/received_events","type":"User","site_admin":false},"assignees":[{"login":"szeke","id":1074599,"node_id":"MDQ6VXNlcjEwNzQ1OTk=","avatar_url":"https://avatars2.githubusercontent.com/u/1074599?v=4","gravatar_id":"","url":"https://api.github.com/users/szeke","html_url":"https://github.com/szeke","followers_url":"https://api.github.com/users/szeke/followers","following_url":"https://api.github.com/users/szeke/following{/other_user}","gists_url":"https://api.github.com/users/szeke/gists{/gist_id}","starred_url":"https://api.github.com/users/szeke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/szeke/subscriptions","organizations_url":"https://api.github.com/users/szeke/orgs","repos_url":"https://api.github.com/users/szeke/repos","events_url":"https://api.github.com/users/szeke/events{/privacy}","received_events_url":"https://api.github.com/users/szeke/received_events","type":"User","site_admin":false},{"login":"RqS","id":6547905,"node_id":"MDQ6VXNlcjY1NDc5MDU=","avatar_url":"https://avatars2.githubusercontent.com/u/6547905?v=4","gravatar_id":"","url":"https://api.github.com/users/RqS","html_url":"https://github.com/RqS","followers_url":"https://api.github.com/users/RqS/followers","following_url":"https://api.github.com/users/RqS/following{/other_user}","gists_url":"https://api.github.com/users/RqS/gists{/gist_id}","starred_url":"https://api.github.com/users/RqS/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RqS/subscriptions","organizations_url":"https://api.github.com/users/RqS/orgs","repos_url":"https://api.github.com/users/RqS/repos","events_url":"https://api.github.com/users/RqS/events{/privacy}","received_events_url":"https://api.github.com/users/RqS/received_events","type":"User","site_admin":false},{"login":"BhavyaGulati","id":7070280,"node_id":"MDQ6VXNlcjcwNzAyODA=","avatar_url":"https://avatars1.githubusercontent.com/u/7070280?v=4","gravatar_id":"","url":"https://api.github.com/users/BhavyaGulati","html_url":"https://github.com/BhavyaGulati","followers_url":"https://api.github.com/users/BhavyaGulati/followers","following_url":"https://api.github.com/users/BhavyaGulati/following{/other_user}","gists_url":"https://api.github.com/users/BhavyaGulati/gists{/gist_id}","starred_url":"https://api.github.com/users/BhavyaGulati/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/BhavyaGulati/subscriptions","organizations_url":"https://api.github.com/users/BhavyaGulati/orgs","repos_url":"https://api.github.com/users/BhavyaGulati/repos","events_url":"https://api.github.com/users/BhavyaGulati/events{/privacy}","received_events_url":"https://api.github.com/users/BhavyaGulati/received_events","type":"User","site_admin":false}],"milestone":null,"comments":0,"created_at":"2018-11-15T23:37:02Z","updated_at":"2018-11-18T22:48:59Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Materialize worldbank data for all time across all countries takes too long to finish. Not able to materialize it within acceptable time.\r\n1. Consider break it down for every country. This means we will have a dataset for all time range for a single country, each associates with one metadata. There is already around 16000 datasets. Doing this will make the number of datasets (metadata) become 16000* (#countries). And we need to en`\r\n2. Otherwise, since the dataset is too big for materializing and profiling, we can put a fixed set of countries in the country column as named_entity. But for materialization, need to query with list of named_entities (countries).\r\n\r\nWe can discuss here for the correct way of dealing with worldbank."},"comment":{"url":"https://api.github.com/repos/usc-isi-i2/datamart/issues/comments/439733143","html_url":"https://github.com/usc-isi-i2/datamart/issues/50#issuecomment-439733143","issue_url":"https://api.github.com/repos/usc-isi-i2/datamart/issues/50","id":439733143,"node_id":"MDEyOklzc3VlQ29tbWVudDQzOTczMzE0Mw==","user":{"login":"szeke","id":1074599,"node_id":"MDQ6VXNlcjEwNzQ1OTk=","avatar_url":"https://avatars2.githubusercontent.com/u/1074599?v=4","gravatar_id":"","url":"https://api.github.com/users/szeke","html_url":"https://github.com/szeke","followers_url":"https://api.github.com/users/szeke/followers","following_url":"https://api.github.com/users/szeke/following{/other_user}","gists_url":"https://api.github.com/users/szeke/gists{/gist_id}","starred_url":"https://api.github.com/users/szeke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/szeke/subscriptions","organizations_url":"https://api.github.com/users/szeke/orgs","repos_url":"https://api.github.com/users/szeke/repos","events_url":"https://api.github.com/users/szeke/events{/privacy}","received_events_url":"https://api.github.com/users/szeke/received_events","type":"User","site_admin":false},"created_at":"2018-11-18T22:48:59Z","updated_at":"2018-11-18T22:48:59Z","author_association":"MEMBER","body":"Let's do it by country. It is OK if the total number is 16000*200 (countries). My guess is that not all indicators are available for every country. @BhavyaGulati please make the changes."}}
{"actor":{"display_login":"szeke"}}