{"action":"created","issue":{"url":"https://api.github.com/repos/ldbc/ldbc_snb_datagen/issues/17","repository_url":"https://api.github.com/repos/ldbc/ldbc_snb_datagen","labels_url":"https://api.github.com/repos/ldbc/ldbc_snb_datagen/issues/17/labels{/name}","comments_url":"https://api.github.com/repos/ldbc/ldbc_snb_datagen/issues/17/comments","events_url":"https://api.github.com/repos/ldbc/ldbc_snb_datagen/issues/17/events","html_url":"https://github.com/ldbc/ldbc_snb_datagen/issues/17","id":134273469,"number":17,"title":"generateparams error when scaling","user":{"login":"MarcoLotz","id":4623265,"avatar_url":"https://avatars.githubusercontent.com/u/4623265?v=3","gravatar_id":"","url":"https://api.github.com/users/MarcoLotz","html_url":"https://github.com/MarcoLotz","followers_url":"https://api.github.com/users/MarcoLotz/followers","following_url":"https://api.github.com/users/MarcoLotz/following{/other_user}","gists_url":"https://api.github.com/users/MarcoLotz/gists{/gist_id}","starred_url":"https://api.github.com/users/MarcoLotz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MarcoLotz/subscriptions","organizations_url":"https://api.github.com/users/MarcoLotz/orgs","repos_url":"https://api.github.com/users/MarcoLotz/repos","events_url":"https://api.github.com/users/MarcoLotz/events{/privacy}","received_events_url":"https://api.github.com/users/MarcoLotz/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-02-17T13:02:27Z","updated_at":"2016-02-18T18:24:48Z","closed_at":null,"body":"Hi there,\r\n\r\nI am trying to generate the 100GB dataset, but I am getting the following error:\r\nI do have enough memory on the YARN containers (set even to 200GB each) and still have this error happening.\r\n\r\n```java\r\nTraceback (most recent call last):\r\n  File \"paramgenerator/generateparams.py\", line 296, in <module>\r\n    sys.exit(main())\r\n  File \"paramgenerator/generateparams.py\", line 152, in main\r\n    selectedPersonParams[i] = discoverparams.generate(factors)\r\n  File \"/home/[user]/ldbc/ldbc_snb_datagen/paramgenerator/discoverparams.py\", line 122, in generate\r\n    params = len(factors[0]) -1\r\n```\r\n\r\nSo far I was able to generate up to the 10GB one. Any ideas of what could this be?"},"comment":{"url":"https://api.github.com/repos/ldbc/ldbc_snb_datagen/issues/comments/185847433","html_url":"https://github.com/ldbc/ldbc_snb_datagen/issues/17#issuecomment-185847433","issue_url":"https://api.github.com/repos/ldbc/ldbc_snb_datagen/issues/17","id":185847433,"user":{"login":"ArnauPrat","id":5836966,"avatar_url":"https://avatars.githubusercontent.com/u/5836966?v=3","gravatar_id":"","url":"https://api.github.com/users/ArnauPrat","html_url":"https://github.com/ArnauPrat","followers_url":"https://api.github.com/users/ArnauPrat/followers","following_url":"https://api.github.com/users/ArnauPrat/following{/other_user}","gists_url":"https://api.github.com/users/ArnauPrat/gists{/gist_id}","starred_url":"https://api.github.com/users/ArnauPrat/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ArnauPrat/subscriptions","organizations_url":"https://api.github.com/users/ArnauPrat/orgs","repos_url":"https://api.github.com/users/ArnauPrat/repos","events_url":"https://api.github.com/users/ArnauPrat/events{/privacy}","received_events_url":"https://api.github.com/users/ArnauPrat/received_events","type":"User","site_admin":false},"created_at":"2016-02-18T18:24:48Z","updated_at":"2016-02-18T18:24:48Z","body":"Hi Marco,\n\nIn my laptop I use 2.7.11. On a hadoop cluster I have 2.7.3, and I have \nsuccessfully run parameter generation in a machine with 2.7.9\n\nCan you confirm that data generation is working properly? Make sure you \npull the latest changes from master branch before executing.\n\nRegards,\n\nArnau\n\nOn 18/02/16 18:18, Marco Aurelio Lotz wrote:\n>\n> Arnau, can you please provide me the exact python version that this \n> datasets have been successfully generated? Have tried versions 2.7.6, \n> 2.7.10 and 2.6.9 and I am getting python exceptions.\n>\n> On 2.6.9 I am getting the following:\n>\n> ^CTraceback (most recent call last):\n>    File\"paramgenerator/generateparams.py\", line3,in  <module>\n>      import  readfactors\n>    File\"/home/lotz/ldbc/ldbc_snb_datagen/paramgenerator/readfactors.py\", line7\n>      FACTOR_MAP  =  {value: keyfor  (key, value)in  enumerate(FACTORS)}\n>\n> —\n> Reply to this email directly or view it on GitHub \n> <https://github.com/ldbc/ldbc_snb_datagen/issues/17#issuecomment-185821610>.\n>\n\n"}}