{"action":"created","issue":{"url":"https://api.github.com/repos/mquad/hgru4rec/issues/10","repository_url":"https://api.github.com/repos/mquad/hgru4rec","labels_url":"https://api.github.com/repos/mquad/hgru4rec/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/mquad/hgru4rec/issues/10/comments","events_url":"https://api.github.com/repos/mquad/hgru4rec/issues/10/events","html_url":"https://github.com/mquad/hgru4rec/issues/10","id":365129752,"node_id":"MDU6SXNzdWUzNjUxMjk3NTI=","number":10,"title":"Incremental Memory","user":{"login":"XSeaty","id":40487989,"node_id":"MDQ6VXNlcjQwNDg3OTg5","avatar_url":"https://avatars3.githubusercontent.com/u/40487989?v=4","gravatar_id":"","url":"https://api.github.com/users/XSeaty","html_url":"https://github.com/XSeaty","followers_url":"https://api.github.com/users/XSeaty/followers","following_url":"https://api.github.com/users/XSeaty/following{/other_user}","gists_url":"https://api.github.com/users/XSeaty/gists{/gist_id}","starred_url":"https://api.github.com/users/XSeaty/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/XSeaty/subscriptions","organizations_url":"https://api.github.com/users/XSeaty/orgs","repos_url":"https://api.github.com/users/XSeaty/repos","events_url":"https://api.github.com/users/XSeaty/events{/privacy}","received_events_url":"https://api.github.com/users/XSeaty/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-09-29T12:51:30Z","updated_at":"2018-12-13T11:39:47Z","closed_at":null,"author_association":"NONE","body":"I add `logger.info(memmory_use)` in the `iterate` function which is called in `HGRU4REC.fit` and get the log below. Is this normal?\r\n## other information\r\nI run this code with CPU.\r\nI run this code on my own dataset (about 28 million interaction)\r\nI notice this because the process is out of memory.\r\n\r\n## training log of the first epoch\r\n2018-09-29 19:57:07,118: __main__: INFO: Training started\r\n2018-09-29 19:57:38,527: hgru4rec: INFO: Epoch Begin!\r\n2018-09-29 19:57:38,530: hgru4rec: INFO: 227.51(GB) memory left.\r\n2018-09-29 19:57:38,600: hgru4rec: INFO: 227.51(GB) memory left.\r\n..........\r\n2018-09-29 20:01:47,200: hgru4rec: INFO: 209.16(GB) memory left.\r\n..........\r\n2018-09-29 20:05:31,439: hgru4rec: INFO: 191.81(GB) memory left.\r\n..........\r\n2018-09-29 20:08:45,621: hgru4rec: INFO: 177.09(GB) memory left.\r\n..........\r\n2018-09-29 20:11:37,108: hgru4rec: INFO: 164.1(GB) memory left.\r\n..........\r\n2018-09-29 20:14:05,733: hgru4rec: INFO: 152.86(GB) memory left.\r\n..........\r\n2018-09-29 20:16:07,679: hgru4rec: INFO: 143.68(GB) memory left.\r\n..........\r\n2018-09-29 20:17:50,016: hgru4rec: INFO: 136.02(GB) memory left.\r\n..........\r\n2018-09-29 20:19:15,243: hgru4rec: INFO: 129.7(GB) memory left.\r\n..........\r\n2018-09-29 20:20:31,730: hgru4rec: INFO: 124.03(GB) memory left.\r\n..........\r\n2018-09-29 20:22:54,691: hgru4rec: INFO: 114.76(GB) memory left.\r\n..........\r\n2018-09-29 20:26:05,653: hgru4rec: INFO: 106.84(GB) memory left.\r\n..........\r\n2018-09-29 20:28:25,798: hgru4rec: INFO: 96.48(GB) memory left.\r\n..........\r\n2018-09-29 20:31:37,315: hgru4rec: INFO: 82.28(GB) memory left.\r\n....\r\n2018-09-29 20:32:39,487: hgru4rec: INFO: Epoch 0 - train cost: 0.8193\r\n"},"comment":{"url":"https://api.github.com/repos/mquad/hgru4rec/issues/comments/446938666","html_url":"https://github.com/mquad/hgru4rec/issues/10#issuecomment-446938666","issue_url":"https://api.github.com/repos/mquad/hgru4rec/issues/10","id":446938666,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0NjkzODY2Ng==","user":{"login":"mquad","id":6952464,"node_id":"MDQ6VXNlcjY5NTI0NjQ=","avatar_url":"https://avatars2.githubusercontent.com/u/6952464?v=4","gravatar_id":"","url":"https://api.github.com/users/mquad","html_url":"https://github.com/mquad","followers_url":"https://api.github.com/users/mquad/followers","following_url":"https://api.github.com/users/mquad/following{/other_user}","gists_url":"https://api.github.com/users/mquad/gists{/gist_id}","starred_url":"https://api.github.com/users/mquad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mquad/subscriptions","organizations_url":"https://api.github.com/users/mquad/orgs","repos_url":"https://api.github.com/users/mquad/repos","events_url":"https://api.github.com/users/mquad/events{/privacy}","received_events_url":"https://api.github.com/users/mquad/received_events","type":"User","site_admin":false},"created_at":"2018-12-13T11:39:47Z","updated_at":"2018-12-13T11:39:47Z","author_association":"OWNER","body":"Nice. Extremely large item sets will likely benefit of an additional embedding layer in input to reduce the memory consumption. Also, if you like, check out the main GRU4Rec repo (https://github.com/hidasib/GRU4Rec) for some updates on speed and mem. consumption that will likely apply to this code base as well."}}
{"actor":{"display_login":"mquad"}}