{"action":"created","issue":{"url":"https://api.github.com/repos/marbl/canu/issues/237","repository_url":"https://api.github.com/repos/marbl/canu","labels_url":"https://api.github.com/repos/marbl/canu/issues/237/labels{/name}","comments_url":"https://api.github.com/repos/marbl/canu/issues/237/comments","events_url":"https://api.github.com/repos/marbl/canu/issues/237/events","html_url":"https://github.com/marbl/canu/issues/237","id":171895252,"number":237,"title":"gzip: stdout: Disk quota exceeded   \\   slurmstepd:  Exceeded job memory limit at some point.","user":{"login":"hermeseduardo","id":16615582,"avatar_url":"https://avatars.githubusercontent.com/u/16615582?v=3","gravatar_id":"","url":"https://api.github.com/users/hermeseduardo","html_url":"https://github.com/hermeseduardo","followers_url":"https://api.github.com/users/hermeseduardo/followers","following_url":"https://api.github.com/users/hermeseduardo/following{/other_user}","gists_url":"https://api.github.com/users/hermeseduardo/gists{/gist_id}","starred_url":"https://api.github.com/users/hermeseduardo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hermeseduardo/subscriptions","organizations_url":"https://api.github.com/users/hermeseduardo/orgs","repos_url":"https://api.github.com/users/hermeseduardo/repos","events_url":"https://api.github.com/users/hermeseduardo/events{/privacy}","received_events_url":"https://api.github.com/users/hermeseduardo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2016-08-18T13:01:14Z","updated_at":"2016-08-21T12:50:04Z","closed_at":null,"body":"Hi there, I ran into some issues while running mhap, the precompute looks OK, any suggestion? bellow the mhap*.out\r\n\r\nHE\r\n\r\n\r\nmhap.229165_10.out:\r\nRunning block 000003 in query 000006\r\nPicked up _JAVA_OPTIONS: -XX:+UseSerialGC\r\nRunning with these settings:\r\n--filter-threshold = 5.0E-6\r\n--help = false\r\n--max-shift = 0.2\r\n--min-olap-length = 116\r\n--min-store-length = 0\r\n--no-self = false\r\n--no-tf = false\r\n--num-hashes = 768\r\n--num-min-matches = 2\r\n--num-threads = 10\r\n--ordered-kmer-size = 12\r\n--ordered-sketch-size = 1536\r\n--repeat-weight = 0.9\r\n--settings = 0\r\n--store-full-id = false\r\n--supress-noise = 0\r\n--threshold = 0.73\r\n--version = false\r\n-f = /flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis-complete/correction/0-mercounts/Cynegetis_complete.ms16.frequentMers.ignore.gz\r\n-h = false\r\n-k = 16\r\n-p = \r\n-q = /flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis-complete/correction/1-overlapper/queries/000006\r\n-s = /flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis-complete/correction/1-overlapper/blocks/000003.dat\r\n\r\nReading in filter file /flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis-complete/correction/0-mercounts/Cynegetis_complete.ms16.frequentMers.ignore.gz.\r\nTime (s) to read filter file: 0.13107088\r\nRead in k-mer filter for sizes: [16]\r\nProcessing files for storage in reverse index...\r\nCurrent # sequences loaded and processed from file: 5000...\r\nCurrent # sequences loaded and processed from file: 10000...\r\nCurrent # sequences loaded and processed from file: 15000...\r\nCurrent # sequences loaded and processed from file: 20000...\r\nCurrent # sequences loaded and processed from file: 25000...\r\nCurrent # sequences loaded and processed from file: 30000...\r\nCurrent # sequences loaded and processed from file: 35000...\r\nCurrent # sequences loaded and processed from file: 40000...\r\nCurrent # sequences loaded and processed from file: 45000...\r\nCurrent # sequences loaded and processed from file: 50000...\r\nCurrent # sequences loaded and processed from file: 55000...\r\nCurrent # sequences loaded and processed from file: 60000...\r\nCurrent # sequences loaded and processed from file: 65000...\r\nCurrent # sequences loaded and processed from file: 70000...\r\nCurrent # sequences loaded and processed from file: 75000...\r\nCurrent # sequences loaded and processed from file: 80000...\r\nCurrent # sequences loaded and processed from file: 85000...\r\nCurrent # sequences loaded and processed from file: 90000...\r\nCurrent # sequences loaded and processed from file: 95000...\r\nCurrent # sequences loaded and processed from file: 100000...\r\nCurrent # sequences loaded and processed from file: 105000...\r\nCurrent # sequences loaded and processed from file: 110000...\r\nCurrent # sequences loaded and processed from file: 115000...\r\nCurrent # sequences loaded and processed from file: 120000...\r\nCurrent # sequences loaded and processed from file: 125000...\r\nCurrent # sequences loaded and processed from file: 130000...\r\nCurrent # sequences loaded and processed from file: 135000...\r\nCurrent # sequences loaded and processed from file: 140000...\r\nCurrent # sequences loaded and processed from file: 145000...\r\nCurrent # sequences loaded and processed from file: 150000...\r\nCurrent # sequences loaded and processed from file: 155000...\r\nCurrent # sequences loaded and processed from file: 160000...\r\nCurrent # sequences loaded and processed from file: 165000...\r\nCurrent # sequences stored: 5000...\r\nCurrent # sequences stored: 10000...\r\nCurrent # sequences stored: 15000...\r\nCurrent # sequences stored: 20000...\r\nCurrent # sequences stored: 25000...\r\nCurrent # sequences stored: 30000...\r\nCurrent # sequences stored: 35000...\r\nCurrent # sequences stored: 40000...\r\nCurrent # sequences stored: 45000...\r\nCurrent # sequences stored: 50000...\r\nCurrent # sequences stored: 55000...\r\nCurrent # sequences stored: 60000...\r\nCurrent # sequences stored: 65000...\r\nCurrent # sequences stored: 70000...\r\nCurrent # sequences stored: 75000...\r\nCurrent # sequences stored: 80000...\r\nCurrent # sequences stored: 85000...\r\nCurrent # sequences stored: 90000...\r\nCurrent # sequences stored: 95000...\r\nCurrent # sequences stored: 100000...\r\nCurrent # sequences stored: 105000...\r\nCurrent # sequences stored: 110000...\r\nCurrent # sequences stored: 115000...\r\nCurrent # sequences stored: 120000...\r\nCurrent # sequences stored: 125000...\r\nCurrent # sequences stored: 130000...\r\nCurrent # sequences stored: 135000...\r\nCurrent # sequences stored: 140000...\r\nCurrent # sequences stored: 145000...\r\nCurrent # sequences stored: 150000...\r\nCurrent # sequences stored: 155000...\r\nCurrent # sequences stored: 160000...\r\nCurrent # sequences stored: 165000...\r\nStored 165000 sequences in the index.\r\nProcessed 165000 unique sequences (fwd and rev).\r\nTime (s) to read and hash from file: 88.458012194\r\nTime (s) to score and output to self: 4752.058701049\r\nOpened fasta file /flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis-complete/correction/1-overlapper/blocks/000004.dat.\r\nCurrent # sequences loaded and processed from file: 5000...\r\nCurrent # sequences loaded and processed from file: 10000...\r\nCurrent # sequences loaded and processed from file: 15000...\r\nCurrent # sequences loaded and processed from file: 20000...\r\nCurrent # sequences loaded and processed from file: 25000...\r\nCurrent # sequences loaded and processed from file: 30000...\r\nCurrent # sequences loaded and processed from file: 35000...\r\nCurrent # sequences loaded and processed from file: 40000...\r\nCurrent # sequences loaded and processed from file: 45000...\r\nCurrent # sequences loaded and processed from file: 50000...\r\nCurrent # sequences loaded and processed from file: 55000...\r\nCurrent # sequences loaded and processed from file: 60000...\r\nCurrent # sequences loaded and processed from file: 65000...\r\nCurrent # sequences loaded and processed from file: 70000...\r\nCurrent # sequences loaded and processed from file: 75000...\r\nCurrent # sequences loaded and processed from file: 80000...\r\nProcessed 82500 to sequences.\r\nTime (s) to score, hash to-file, and output: 9006.381839581001\r\nOpened fasta file /flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis-complete/correction/1-overlapper/blocks/000005.dat.\r\nCurrent # sequences loaded and processed from file: 5000...\r\nCurrent # sequences loaded and processed from file: 10000...\r\nCurrent # sequences loaded and processed from file: 15000...\r\nCurrent # sequences loaded and processed from file: 20000...\r\nCurrent # sequences loaded and processed from file: 25000...\r\nCurrent # sequences loaded and processed from file: 30000...\r\nCurrent # sequences loaded and processed from file: 35000...\r\nCurrent # sequences loaded and processed from file: 40000...\r\nCurrent # sequences loaded and processed from file: 45000...\r\nCurrent # sequences loaded and processed from file: 50000...\r\nCurrent # sequences loaded and processed from file: 55000...\r\nCurrent # sequences loaded and processed from file: 60000...\r\nCurrent # sequences loaded and processed from file: 65000...\r\nCurrent # sequences loaded and processed from file: 70000...\r\nCurrent # sequences loaded and processed from file: 75000...\r\nCurrent # sequences loaded and processed from file: 80000...\r\nProcessed 82500 to sequences.\r\nTime (s) to score, hash to-file, and output: 9487.53588757\r\nTotal scoring time (s): 23246.012313047002\r\nTotal time (s): 23334.470375489\r\nMinHash search time (s): 6488.403775832\r\nTotal matches found: 1491753232\r\nAverage number of matches per lookup: 6027.285785858586\r\nAverage number of table elements processed per lookup: 59279.971438383836\r\nAverage number of table elements processed per match: 9.835268069994033\r\nAverage % of hashed sequences hit per lookup: 21.894370385062746\r\nAverage % of hashed sequences hit that are matches: 16.68419969167093\r\nAverage % of hashed sequences fully compared that are matches: 50.45109665988456\r\n\r\ngzip: stdout: Disk quota exceeded\r\nslurmstepd: Exceeded step memory limit at some point.\r\nslurmstepd: Exceeded job memory limit at some point."},"comment":{"url":"https://api.github.com/repos/marbl/canu/issues/comments/241255545","html_url":"https://github.com/marbl/canu/issues/237#issuecomment-241255545","issue_url":"https://api.github.com/repos/marbl/canu/issues/237","id":241255545,"user":{"login":"hermeseduardo","id":16615582,"avatar_url":"https://avatars.githubusercontent.com/u/16615582?v=3","gravatar_id":"","url":"https://api.github.com/users/hermeseduardo","html_url":"https://github.com/hermeseduardo","followers_url":"https://api.github.com/users/hermeseduardo/followers","following_url":"https://api.github.com/users/hermeseduardo/following{/other_user}","gists_url":"https://api.github.com/users/hermeseduardo/gists{/gist_id}","starred_url":"https://api.github.com/users/hermeseduardo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hermeseduardo/subscriptions","organizations_url":"https://api.github.com/users/hermeseduardo/orgs","repos_url":"https://api.github.com/users/hermeseduardo/repos","events_url":"https://api.github.com/users/hermeseduardo/events{/privacy}","received_events_url":"https://api.github.com/users/hermeseduardo/received_events","type":"User","site_admin":false},"created_at":"2016-08-21T12:50:04Z","updated_at":"2016-08-21T12:50:04Z","body":"looks like this line (305) in mhap.sh  is generating the segmentation fault:\r\n\r\nfi\r\nif [ -e \"/flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis/correction/1-overlapper/results/$qry.mhap.ovb.gz\" ] ; then\r\n  mv -f \"/flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis/correction/1-overlapper/results/$qry.mhap.ovb.gz\" \"/flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis/correction/1-overlapper/results/$qry.ovb.gz\"\r\nfi\r\n\r\nhere the ouput from mhap:\r\n\r\nRunning block 000003 in query 000011\r\n/flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis/correction/1-overlapper/mhap.sh: line 305: 134487 Segmentation fault      $bin/mhapConvert $cvt -o /flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis/correction/1-overlapper/results/$qry.mhap.ovb.WORKING.gz /flush1/esc003/Cynegetis_genome_transcriptome/Cynegetis_impunctata_WORKING_FOLDER/canu_assembly/Cynegetis/correction/1-overlapper/results/$qry.mhap\r\n\r\n"}}
{"actor":{"display_login":"hermeseduardo"}}