{"action":"created","issue":{"url":"https://api.github.com/repos/mlpack/mlpack/issues/1555","repository_url":"https://api.github.com/repos/mlpack/mlpack","labels_url":"https://api.github.com/repos/mlpack/mlpack/issues/1555/labels{/name}","comments_url":"https://api.github.com/repos/mlpack/mlpack/issues/1555/comments","events_url":"https://api.github.com/repos/mlpack/mlpack/issues/1555/events","html_url":"https://github.com/mlpack/mlpack/issues/1555","id":377736097,"node_id":"MDU6SXNzdWUzNzc3MzYwOTc=","number":1555,"title":"Last step of mlpack_cf takes forever to finish","user":{"login":"stiv-yakovenko","id":4163254,"node_id":"MDQ6VXNlcjQxNjMyNTQ=","avatar_url":"https://avatars0.githubusercontent.com/u/4163254?v=4","gravatar_id":"","url":"https://api.github.com/users/stiv-yakovenko","html_url":"https://github.com/stiv-yakovenko","followers_url":"https://api.github.com/users/stiv-yakovenko/followers","following_url":"https://api.github.com/users/stiv-yakovenko/following{/other_user}","gists_url":"https://api.github.com/users/stiv-yakovenko/gists{/gist_id}","starred_url":"https://api.github.com/users/stiv-yakovenko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stiv-yakovenko/subscriptions","organizations_url":"https://api.github.com/users/stiv-yakovenko/orgs","repos_url":"https://api.github.com/users/stiv-yakovenko/repos","events_url":"https://api.github.com/users/stiv-yakovenko/events{/privacy}","received_events_url":"https://api.github.com/users/stiv-yakovenko/received_events","type":"User","site_admin":false},"labels":[{"id":164372968,"node_id":"MDU6TGFiZWwxNjQzNzI5Njg=","url":"https://api.github.com/repos/mlpack/mlpack/labels/D:%20easy","name":"D: easy","color":"207de5","default":false},{"id":161438265,"node_id":"MDU6TGFiZWwxNjE0MzgyNjU=","url":"https://api.github.com/repos/mlpack/mlpack/labels/T:%20enhancement","name":"T: enhancement","color":"cccccc","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2018-11-06T08:20:45Z","updated_at":"2018-11-13T19:36:46Z","closed_at":null,"author_association":"NONE","body":"My data file is few gb in size, 30M user-item pairs, 4M users, ~300K items, calculating each iteration takes reasonable time, few minutes:\r\n\r\n    ~/mlpack-3.0.3/build/bin/mlpack_cf -A -t oryxlog.int.txt -o rec.txt -v -R 40 -c 200\r\n\r\n![image](https://user-images.githubusercontent.com/4163254/48051166-f4233880-e1d6-11e8-8570-d33fcebe1f41.png)\r\n\r\nbut this last step has already been in progress for ~5hrs. It is taking just 300% of cpu (according to **top** linux command), but when calculation is in the middle, then I see 800% cpu consumption. How can I make it faster? What is it doing under the hood?"},"comment":{"url":"https://api.github.com/repos/mlpack/mlpack/issues/comments/438407235","html_url":"https://github.com/mlpack/mlpack/issues/1555#issuecomment-438407235","issue_url":"https://api.github.com/repos/mlpack/mlpack/issues/1555","id":438407235,"node_id":"MDEyOklzc3VlQ29tbWVudDQzODQwNzIzNQ==","user":{"login":"stiv-yakovenko","id":4163254,"node_id":"MDQ6VXNlcjQxNjMyNTQ=","avatar_url":"https://avatars0.githubusercontent.com/u/4163254?v=4","gravatar_id":"","url":"https://api.github.com/users/stiv-yakovenko","html_url":"https://github.com/stiv-yakovenko","followers_url":"https://api.github.com/users/stiv-yakovenko/followers","following_url":"https://api.github.com/users/stiv-yakovenko/following{/other_user}","gists_url":"https://api.github.com/users/stiv-yakovenko/gists{/gist_id}","starred_url":"https://api.github.com/users/stiv-yakovenko/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/stiv-yakovenko/subscriptions","organizations_url":"https://api.github.com/users/stiv-yakovenko/orgs","repos_url":"https://api.github.com/users/stiv-yakovenko/repos","events_url":"https://api.github.com/users/stiv-yakovenko/events{/privacy}","received_events_url":"https://api.github.com/users/stiv-yakovenko/received_events","type":"User","site_admin":false},"created_at":"2018-11-13T19:36:46Z","updated_at":"2018-11-13T19:36:46Z","author_association":"NONE","body":"And one more comment to be clear: my code is of cause incomplete, but I just wanted to show that even brute force non-optimized iteration over all items and users performs faster (~1h) then current mlpack_cf (~5hr) on large dataset. May be optimisation is redundant, or may be it can be implemented even faster then my 1h."}}
{"actor":{"display_login":"stiv-yakovenko"}}