{"action":"created","issue":{"url":"https://api.github.com/repos/trinityrnaseq/trinityrnaseq/issues/116","repository_url":"https://api.github.com/repos/trinityrnaseq/trinityrnaseq","labels_url":"https://api.github.com/repos/trinityrnaseq/trinityrnaseq/issues/116/labels{/name}","comments_url":"https://api.github.com/repos/trinityrnaseq/trinityrnaseq/issues/116/comments","events_url":"https://api.github.com/repos/trinityrnaseq/trinityrnaseq/issues/116/events","html_url":"https://github.com/trinityrnaseq/trinityrnaseq/issues/116","id":144253009,"number":116,"title":"Failed butterfly commands: java.lang.OutOfMemoryError","user":{"login":"ricardojp","id":7750182,"avatar_url":"https://avatars.githubusercontent.com/u/7750182?v=3","gravatar_id":"","url":"https://api.github.com/users/ricardojp","html_url":"https://github.com/ricardojp","followers_url":"https://api.github.com/users/ricardojp/followers","following_url":"https://api.github.com/users/ricardojp/following{/other_user}","gists_url":"https://api.github.com/users/ricardojp/gists{/gist_id}","starred_url":"https://api.github.com/users/ricardojp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ricardojp/subscriptions","organizations_url":"https://api.github.com/users/ricardojp/orgs","repos_url":"https://api.github.com/users/ricardojp/repos","events_url":"https://api.github.com/users/ricardojp/events{/privacy}","received_events_url":"https://api.github.com/users/ricardojp/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2016-03-29T12:39:44Z","updated_at":"2016-04-13T00:19:37Z","closed_at":null,"body":"Dear all,\r\n\r\nI'm running Trinity (v 2.2.0) on a cluster linux debian with 500 G of RAM.\r\nI'm running 10 paired end samples separately. 8 of those ran fine, but the 2 largest samples (28 G for the two files together) get errors during the Butterfly step.\r\n\r\nThe standard command that worked for most samples uses --CPU 15 --max_memory 40G. But for those samples I get errors related to Memory (\"java.lang.OutOfMemoryError: GC overhead limit exceeded\" or \"java.lang.OutOfMemoryError: Java heap space\").\r\n\r\nAfter reading the [FAQ](https://github.com/trinityrnaseq/trinityrnaseq/wiki/Trinity-FAQ#ques_butterfly_GC_thread_fail) I have reduced the --CPU settings to 10 and 5, with variations of --max_memory, and always got the same error...\r\n\r\nThe file FailedCommands shows:\r\n/usr/local/src/trinity/2.2.0/util/support_scripts/../../Trinity --single \"/emc/data/pereira/chort/assemblies/sample7/trinity/read_partitions/Fb_0/CBin_622/c62242.trinity.reads.fa\" --output \"/emc/data/pereira/chort/assemblies/sample7/trinity/read_partitions/Fb_0/CBin_622/c62242.trinity.reads.fa.out\" --CPU 1 --max_memory 1G --seqType fa --trinity_complete --full_cleanup\r\nI found that the --CPU and  --max_memory settings strange, since when calling Trinity I had specified --CPU 15 --max_memory 40G.\r\nI have ran the failed command exactly as in the FailedCommands file, and got the same error as expected. Surprisingly, when I changed it to --CPU 1 --max_memory 10G, it still ran using only 1G, and gave me the same error message...\r\n\r\nIs there any chance that the CPU and max_memory settings evoked with Trinity are not carried on during Butterfly?\r\n\r\nI would really appreciate any lead on this!\r\nCheers,\r\nr"},"comment":{"url":"https://api.github.com/repos/trinityrnaseq/trinityrnaseq/issues/comments/209161184","html_url":"https://github.com/trinityrnaseq/trinityrnaseq/issues/116#issuecomment-209161184","issue_url":"https://api.github.com/repos/trinityrnaseq/trinityrnaseq/issues/116","id":209161184,"user":{"login":"brianjohnhaas","id":7542111,"avatar_url":"https://avatars.githubusercontent.com/u/7542111?v=3","gravatar_id":"","url":"https://api.github.com/users/brianjohnhaas","html_url":"https://github.com/brianjohnhaas","followers_url":"https://api.github.com/users/brianjohnhaas/followers","following_url":"https://api.github.com/users/brianjohnhaas/following{/other_user}","gists_url":"https://api.github.com/users/brianjohnhaas/gists{/gist_id}","starred_url":"https://api.github.com/users/brianjohnhaas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/brianjohnhaas/subscriptions","organizations_url":"https://api.github.com/users/brianjohnhaas/orgs","repos_url":"https://api.github.com/users/brianjohnhaas/repos","events_url":"https://api.github.com/users/brianjohnhaas/events{/privacy}","received_events_url":"https://api.github.com/users/brianjohnhaas/received_events","type":"User","site_admin":false},"created_at":"2016-04-13T00:19:37Z","updated_at":"2016-04-13T00:19:37Z","body":"When another user encountered this, what he ended up doing was just running\nthe normalization on that corresponding c124547.trinity.reads.fa file and\nreplace it with the normalized version.  After deleting the current\ntrinity_out_dir/read_partitions/Fb_1/CBin_1245/c124547.trinity.reads.fa.out\n and then rerunning the original Trinity command, it should process that\nentry just fine and then wrap up.  I'm assuming it was just that one\npartitioned read set that was causing trouble. If there are others, then\nyou'd do the same thing for those as well.\n\nThe alternative approach (in Trinity v2) would be to run that Butterfly\ncommand directly giving it the higher RAM setting, and then to tack the\noriginal failed butterfly command onto the corresponding\n'butterfly_commands.completed' file in that same directory.   When Trinity\nis re-run, it should resume where it left off and wrap up.\n\nBut in general, to avoid these kinds of problems, it's always best to run\nthe normalization when you have a large data set.  I think we might just\nmake normalization the default setting in the next release to minimize\nthese sorts of problems.\n\nbest,\n\n~b\n\nOn Tue, Apr 12, 2016 at 7:24 PM, chrisbala10 <notifications@github.com>\nwrote:\n>\n> Hi All, we've run into a similar problem, and not having much luck with\nthis workaround.\n> What are the modified butterfly commands supposed to yield in terms of\noutput files?\n> The modified butterfly commands are finishing without error:\n>\n> java -Xmx40G -Xms1G -XX:ParallelGCThreads=20 -jar\n/home/cbala/trinityrnaseq-2.2.0/Butterfly/Butterfly.jar -N 100000 -L 200 -F\n500 -C\n/media/BigArseRAID/SummersLab/trimmed_pairedends/trinity_out_dir/read_partitions/Fb_1/CBin_1245/c124547.trinity.reads.fa.out/chrysalis/Component_bins/Cbin0/c0.graph\n--path_reinforcement_distance=25\n>\n> Yet, I can't see any new output files.\n>\n> files like c0.graph.allProbPaths.fasta have filesize=0\n>\n> I've added the rerun failed commands to the\nrecursive_trinity.cmd.completed, but Trinity ignores these added lines, I'm\nguessing because the butterfly commands have not actually been finished?\n>\n> Any suggestions Brian? Matt? anyone? Also trying running w/ normalization\nb/c dataset is pretty big. Don't know if that might help...\n>\n> —\n> You are receiving this because you were mentioned.\n> Reply to this email directly or view it on GitHub\n\n\n\n\n--\n--\nBrian J. Haas\nThe Broad Institute\nhttp://broadinstitute.org/~bhaas\n"}}