{"action":"created","issue":{"url":"https://api.github.com/repos/amznlabs/amazon-dsstne/issues/24","repository_url":"https://api.github.com/repos/amznlabs/amazon-dsstne","labels_url":"https://api.github.com/repos/amznlabs/amazon-dsstne/issues/24/labels{/name}","comments_url":"https://api.github.com/repos/amznlabs/amazon-dsstne/issues/24/comments","events_url":"https://api.github.com/repos/amznlabs/amazon-dsstne/issues/24/events","html_url":"https://github.com/amznlabs/amazon-dsstne/issues/24","id":155986721,"number":24,"title":"Low ranking accuracy of the example with MovieLens20M?","user":{"login":"saulvargas","id":2616433,"avatar_url":"https://avatars.githubusercontent.com/u/2616433?v=3","gravatar_id":"","url":"https://api.github.com/users/saulvargas","html_url":"https://github.com/saulvargas","followers_url":"https://api.github.com/users/saulvargas/followers","following_url":"https://api.github.com/users/saulvargas/following{/other_user}","gists_url":"https://api.github.com/users/saulvargas/gists{/gist_id}","starred_url":"https://api.github.com/users/saulvargas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saulvargas/subscriptions","organizations_url":"https://api.github.com/users/saulvargas/orgs","repos_url":"https://api.github.com/users/saulvargas/repos","events_url":"https://api.github.com/users/saulvargas/events{/privacy}","received_events_url":"https://api.github.com/users/saulvargas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-05-20T15:17:07Z","updated_at":"2016-05-23T16:24:21Z","closed_at":null,"body":"Hi,\r\n\r\nI've been playing around today with DSSTNE with the goal of running the example with MovieLens20M and compare the NN in the example with some state-of-the-art CF algorithms that I have implemented [here](https://github.com/RankSys/RankSys). From my evaluation (which is by no means exhaustive or perfect) the example provided by DSSTNE does not seem to be competitive with respect to state of the art CF algorithms.\r\n\r\nTo summarise, I have downloaded the original MovieLens 20M dataset and I have performed a random 80%-20% partition. I have transformed the training subset to the DSSTNE format, with the only difference that I do not include the timestamps of the dataset, but 1's for all movies (is this actually very important??). I have generated recommendations with my CF algorithms (popularity, user-based and matrix factorisation) and, following the steps in the example, the predictions of DSSTNE. Finally, I have evaluated the performance with the testing subset using precision at cutoff 10.\r\n\r\nThese are the results, the configuration provided in your example does not seem to work very well:\r\npop\t0.10974162112149495\r\nub\t0.24097987334078072\r\nmf\t0.25135912784469483\r\ndsstne\t0.056956854920365056\r\n\r\nI am no expert in ANN's so I cannot figure out easily whether I should modify the parameters in the config.json provided in the example to make it work better. Have you compared the performance of the example with similar CF algorithms? If so, could you please share some results/insights?\r\n\r\nCheers\r\nSaúl\r\n"},"comment":{"url":"https://api.github.com/repos/amznlabs/amazon-dsstne/issues/comments/221022756","html_url":"https://github.com/amznlabs/amazon-dsstne/issues/24#issuecomment-221022756","issue_url":"https://api.github.com/repos/amznlabs/amazon-dsstne/issues/24","id":221022756,"user":{"login":"rgeorgej","id":18041105,"avatar_url":"https://avatars.githubusercontent.com/u/18041105?v=3","gravatar_id":"","url":"https://api.github.com/users/rgeorgej","html_url":"https://github.com/rgeorgej","followers_url":"https://api.github.com/users/rgeorgej/followers","following_url":"https://api.github.com/users/rgeorgej/following{/other_user}","gists_url":"https://api.github.com/users/rgeorgej/gists{/gist_id}","starred_url":"https://api.github.com/users/rgeorgej/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rgeorgej/subscriptions","organizations_url":"https://api.github.com/users/rgeorgej/orgs","repos_url":"https://api.github.com/users/rgeorgej/repos","events_url":"https://api.github.com/users/rgeorgej/events{/privacy}","received_events_url":"https://api.github.com/users/rgeorgej/received_events","type":"User","site_admin":false},"created_at":"2016-05-23T16:24:21Z","updated_at":"2016-05-23T16:24:21Z","body":"@saulvargas is it possible to get us the scripts which you used to generate your test and train dataset"}}
{"actor":{"display_login":"rgeorgej"}}