{"action":"created","issue":{"url":"https://api.github.com/repos/bcgsc/abyss/issues/220","repository_url":"https://api.github.com/repos/bcgsc/abyss","labels_url":"https://api.github.com/repos/bcgsc/abyss/issues/220/labels{/name}","comments_url":"https://api.github.com/repos/bcgsc/abyss/issues/220/comments","events_url":"https://api.github.com/repos/bcgsc/abyss/issues/220/events","html_url":"https://github.com/bcgsc/abyss/issues/220","id":318607391,"number":220,"title":"Choosing the best genome assembly","user":{"login":"mmokrejs","id":3297852,"avatar_url":"https://avatars1.githubusercontent.com/u/3297852?v=4","gravatar_id":"","url":"https://api.github.com/users/mmokrejs","html_url":"https://github.com/mmokrejs","followers_url":"https://api.github.com/users/mmokrejs/followers","following_url":"https://api.github.com/users/mmokrejs/following{/other_user}","gists_url":"https://api.github.com/users/mmokrejs/gists{/gist_id}","starred_url":"https://api.github.com/users/mmokrejs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmokrejs/subscriptions","organizations_url":"https://api.github.com/users/mmokrejs/orgs","repos_url":"https://api.github.com/users/mmokrejs/repos","events_url":"https://api.github.com/users/mmokrejs/events{/privacy}","received_events_url":"https://api.github.com/users/mmokrejs/received_events","type":"User","site_admin":false},"labels":[{"id":1640848,"url":"https://api.github.com/repos/bcgsc/abyss/labels/question","name":"question","color":"cc317c","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":9,"created_at":"2018-04-28T06:57:58Z","updated_at":"2018-04-28T17:52:52Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Hi,\r\n  I am moving my posts from a thread which I somewhat hijacked in the past (https://github.com/bcgsc/abyss/issues/178) for easier reading. I will add the comments of @sjackman here too.\r\n\r\nInteresting, the k=128 assembly went better IMO than the k=64 assembly. These are the contigs but I think it is correct to check those well-assembled segments before scaffodling happens. What am I missing here? Do you think the counts are larger because the k=128 assembly contains more redundant contigs? Was our discussion only about sealing gaps?\r\n\r\n```\r\n$ BBmap-37.90/stats.sh in=abyss_k64/tt_16D1C3L12__abyss_64-3.fa \r\nA\tC\tG\tT\tN\tIUPAC\tOther\tGC\tGC_stdev\r\n0.3026\t0.1979\t0.1980\t0.3014\t0.0000\t0.0000\t0.0000\t0.3959\t0.1092\r\n\r\nMain genome scaffold total:         \t8970038\r\nMain genome contig total:           \t8970038\r\nMain genome scaffold sequence total:\t1545.997 MB\r\nMain genome contig sequence total:  \t1545.997 MB  \t0.000% gap\r\nMain genome scaffold N/L50:         \t984481/207\r\nMain genome contig N/L50:           \t984481/207\r\nMain genome scaffold N/L90:         \t6786697/66\r\nMain genome contig N/L90:           \t6786697/66\r\nMax scaffold length:                \t48.544 KB\r\nMax contig length:                  \t48.544 KB\r\nNumber of scaffolds > 50 KB:        \t0\r\n% main genome in scaffolds > 50 KB: \t0.00%\r\n\r\n\r\nMinimum \tNumber        \tNumber        \tTotal         \tTotal         \tScaffold\r\nScaffold\tof            \tof            \tScaffold      \tContig        \tContig  \r\nLength  \tScaffolds     \tContigs       \tLength        \tLength        \tCoverage\r\n--------\t--------------\t--------------\t--------------\t--------------\t--------\r\n    All \t     8,970,038\t     8,970,038\t 1,545,997,227\t 1,545,997,227\t 100.00%\r\n     50 \t     8,970,038\t     8,970,038\t 1,545,997,227\t 1,545,997,227\t 100.00%\r\n    100 \t     4,217,230\t     4,217,230\t 1,211,849,941\t 1,211,849,941\t 100.00%\r\n    250 \t       763,487\t       763,487\t   724,479,476\t   724,479,476\t 100.00%\r\n    500 \t       386,281\t       386,281\t   597,014,844\t   597,014,844\t 100.00%\r\n   1 KB \t       218,620\t       218,620\t   479,261,380\t   479,261,380\t 100.00%\r\n 2.5 KB \t        56,120\t        56,120\t   220,798,793\t   220,798,793\t 100.00%\r\n   5 KB \t         9,411\t         9,411\t    66,580,089\t    66,580,089\t 100.00%\r\n  10 KB \t           875\t           875\t    11,287,016\t    11,287,016\t 100.00%\r\n  25 KB \t            12\t            12\t       371,039\t       371,039\t 100.00%\r\n\r\n$ BBmap-37.90/stats.sh in=abyss_k128/tt_16D1C3L12__abyss_128-3.fa \r\nA\tC\tG\tT\tN\tIUPAC\tOther\tGC\tGC_stdev\r\n0.3002\t0.2001\t0.2002\t0.2995\t0.0000\t0.0000\t0.0000\t0.4003\t0.0996\r\n\r\nMain genome scaffold total:         \t3835048\r\nMain genome contig total:           \t3835048\r\nMain genome scaffold sequence total:\t1651.869 MB\r\nMain genome contig sequence total:  \t1651.869 MB  \t0.000% gap\r\nMain genome scaffold N/L50:         \t280737/938\r\nMain genome contig N/L50:           \t280737/938\r\nMain genome scaffold N/L90:         \t2578210/142\r\nMain genome contig N/L90:           \t2578210/142\r\nMax scaffold length:                \t56.412 KB\r\nMax contig length:                  \t56.412 KB\r\nNumber of scaffolds > 50 KB:        \t1\r\n% main genome in scaffolds > 50 KB: \t0.00%\r\n\r\n\r\nMinimum \tNumber        \tNumber        \tTotal         \tTotal         \tScaffold\r\nScaffold\tof            \tof            \tScaffold      \tContig        \tContig  \r\nLength  \tScaffolds     \tContigs       \tLength        \tLength        \tCoverage\r\n--------\t--------------\t--------------\t--------------\t--------------\t--------\r\n    All \t     3,835,048\t     3,835,048\t 1,651,869,106\t 1,651,869,106\t 100.00%\r\n    100 \t     3,835,048\t     3,835,048\t 1,651,869,106\t 1,651,869,106\t 100.00%\r\n    250 \t     1,733,198\t     1,733,198\t 1,333,132,564\t 1,333,132,564\t 100.00%\r\n    500 \t       475,698\t       475,698\t   956,024,894\t   956,024,894\t 100.00%\r\n   1 KB \t       267,415\t       267,415\t   813,134,944\t   813,134,944\t 100.00%\r\n 2.5 KB \t       116,134\t       116,134\t   571,167,119\t   571,167,119\t 100.00%\r\n   5 KB \t        38,065\t        38,065\t   299,800,293\t   299,800,293\t 100.00%\r\n  10 KB \t         6,544\t         6,544\t    88,583,607\t    88,583,607\t 100.00%\r\n  25 KB \t           125\t           125\t     3,756,838\t     3,756,838\t 100.00%\r\n  50 KB \t             1\t             1\t        56,412\t        56,412\t 100.00%\r\n\r\n$\r\n```\r\n\r\nAnd when I compare the final scaffolds:\r\n\r\n```\r\n$ BBmap-37.90/stats.sh in=abyss_k64/tt_16D1C3L12__abyss_64-8.fa\r\nA\tC\tG\tT\tN\tIUPAC\tOther\tGC\tGC_stdev\r\n0.3023\t0.1982\t0.1984\t0.3011\t0.1032\t0.0001\t0.0000\t0.3966\t0.1107\r\n\r\nMain genome scaffold total:         \t8066430\r\nMain genome contig total:           \t8196768\r\nMain genome scaffold sequence total:\t1675.455 MB\r\nMain genome contig sequence total:  \t1502.500 MB  \t10.323% gap\r\nMain genome scaffold N/L50:         \t276947/388\r\nMain genome contig N/L50:           \t703390/232\r\nMain genome scaffold N/L90:         \t5555783/69\r\nMain genome contig N/L90:           \t6079298/67\r\nMax scaffold length:                \t369.235 KB\r\nMax contig length:                  \t59.732 KB\r\nNumber of scaffolds > 50 KB:        \t2906\r\n% main genome in scaffolds > 50 KB: \t15.71%\r\n\r\n\r\nMinimum \tNumber        \tNumber        \tTotal         \tTotal         \tScaffold\r\nScaffold\tof            \tof            \tScaffold      \tContig        \tContig  \r\nLength  \tScaffolds     \tContigs       \tLength        \tLength        \tCoverage\r\n--------\t--------------\t--------------\t--------------\t--------------\t--------\r\n    All \t     8,066,430\t     8,196,768\t 1,675,455,273\t 1,502,499,899\t  89.68%\r\n     50 \t     8,066,430\t     8,196,768\t 1,675,455,273\t 1,502,499,899\t  89.68%\r\n    100 \t     3,838,952\t     3,969,290\t 1,376,296,357\t 1,203,340,983\t  87.43%\r\n    250 \t       515,159\t       645,453\t   910,327,281\t   737,374,137\t  81.00%\r\n    500 \t       203,301\t       333,571\t   805,706,044\t   632,758,384\t  78.53%\r\n   1 KB \t        75,439\t       205,708\t   715,848,362\t   542,901,177\t  75.84%\r\n 2.5 KB \t        34,397\t       160,621\t   648,229,490\t   475,601,705\t  73.37%\r\n   5 KB \t        25,225\t       149,598\t   618,496,895\t   446,269,819\t  72.15%\r\n  10 KB \t        16,226\t       128,799\t   555,879,949\t   400,660,524\t  72.08%\r\n  25 KB \t         7,213\t        90,509\t   413,531,573\t   302,275,101\t  73.10%\r\n  50 KB \t         2,906\t        54,936\t   263,145,032\t   196,921,549\t  74.83%\r\n 100 KB \t           806\t        23,619\t   118,660,722\t    91,293,900\t  76.94%\r\n 250 KB \t            36\t         1,952\t    10,260,581\t     8,133,203\t  79.27%\r\n\r\n\r\n$ BBmap-37.90/stats.sh in=tt_16D1C3L12__abyss_k128-8.fa\r\nA\tC\tG\tT\tN\tIUPAC\tOther\tGC\tGC_stdev\r\n0.3001\t0.2002\t0.2003\t0.2994\t0.0776\t0.0001\t0.0000\t0.4005\t0.1032\r\n\r\nMain genome scaffold total:         \t2876731\r\nMain genome contig total:           \t3015816\r\nMain genome scaffold sequence total:\t1703.964 MB\r\nMain genome contig sequence total:  \t1571.653 MB  \t7.765% gap\r\nMain genome scaffold N/L50:         \t11981/19.507 KB\r\nMain genome contig N/L50:           \t108612/2.736 KB\r\nMain genome scaffold N/L90:         \t1639503/171\r\nMain genome contig N/L90:           \t1867631/160\r\nMax scaffold length:                \t873.806 KB\r\nMax contig length:                  \t90.766 KB\r\nNumber of scaffolds > 50 KB:        \t5362\r\n% main genome in scaffolds > 50 KB: \t37.88%\r\n\r\n\r\nMinimum \tNumber        \tNumber        \tTotal         \tTotal         \tScaffold\r\nScaffold\tof            \tof            \tScaffold      \tContig        \tContig  \r\nLength  \tScaffolds     \tContigs       \tLength        \tLength        \tCoverage\r\n--------\t--------------\t--------------\t--------------\t--------------\t--------\r\n    All \t     2,876,731\t     3,015,816\t 1,703,964,196\t 1,571,652,579\t  92.24%\r\n    100 \t     2,876,731\t     3,015,816\t 1,703,964,196\t 1,571,652,579\t  92.24%\r\n    250 \t     1,268,766\t     1,407,851\t 1,454,714,840\t 1,322,403,223\t  90.90%\r\n    500 \t       189,163\t       328,132\t 1,135,210,242\t 1,002,913,588\t  88.35%\r\n   1 KB \t        52,595\t       191,516\t 1,042,703,436\t   910,423,098\t  87.31%\r\n 2.5 KB \t        32,095\t       169,028\t 1,006,465,234\t   874,336,179\t  86.87%\r\n   5 KB \t        23,821\t       159,541\t   978,974,988\t   846,946,092\t  86.51%\r\n  10 KB \t        17,880\t       148,430\t   935,974,349\t   809,391,570\t  86.48%\r\n  25 KB \t         9,965\t       122,063\t   807,426,899\t   701,763,980\t  86.91%\r\n  50 KB \t         5,362\t        93,327\t   645,398,771\t   565,717,262\t  87.65%\r\n 100 KB \t         2,428\t        61,150\t   440,176,973\t   389,086,844\t  88.39%\r\n 250 KB \t           382\t        17,275\t   130,980,817\t   117,113,064\t  89.41%\r\n 500 KB \t            28\t         2,206\t    17,115,130\t    15,343,274\t  89.65%\r\n\r\n```\r\n\r\n@sjackman commented on Feb 8\r\n\r\n> k=64 Main genome contig N/L50:           \t984481/207\r\n> k=128 Main genome contig N/L50:           \t280737/938\r\n\r\n`k=64` is the more contiguous assembly. Try `k=80,96,112`.\r\nI'd suggest using ntCard and GenomeScope first before assembling with ABySS to get a sense of the coverage and k-mer distribution of your data.\r\n\r\n\r\n@mmokrejs  commented on Feb 8\r\nThose numbers were from step -3. In final scaffolds in -8 files there is:\r\n\r\n> k=64 Main genome contig N/L50:           \t703390/232\r\n> k=128 Main genome contig N/L50:           \t108612/2.736 KB\r\n\r\nThose KB are certainly not kilobytes, it is just weird.\r\n\r\n\r\n@sjackman commented on Feb 8\r\nPerhaps it's a a bug in `BBmap-37.90/stats.sh` I'm guessing it should be `2,736`.\r\n`k=64` is more contiguous for both unitigs and scaffolds.\r\n\r\n\r\n@mmokrejs commented on Feb 8\r\nI asked Brian Bushnell for meaning of the MB and KB in the output, here is the answer.\r\n\r\n_Indeed, those are Kbp and Mbp; I should change them. You can add the flag \"format=2\" to print whole numbers of bp with no commas or decimals, instead of Kbp or Mbp. Also, note that \"L50\" for stats.sh means a Length and \"N50\" means a Number of contigs, so the L50 (length of 50th percentile contig) increased from 232 to 2736 in that case, when K increased, meaning continuity improved. There are some programs that have L50 and N50 reversed._\r\n\r\n\r\n@sjackman commented on Feb 9\r\nI'm afraid Brian has the definition of N50 and L50 reversed. It's a common misunderstanding due to an unfortunate nomenclature. N50 is the length of the contig, and L50 is the number of the contigs whose size is N50 or larger. Yes it's weird, but that's the way it is.\r\nSee https://en.wikipedia.org/wiki/N50,_L50,_and_related_statistics\r\nand http://quast.bioinf.spbau.ru/manual.html#sec3.1.1\r\n\r\n\r\n@mmokrejs commented on Feb 11\r\nYou asked for the numbers from `abyss-fac` so that we do not have to think of what Brian has eventually swapped. The mapper used was the abyss-map (I sticked to the default so far).\r\n\r\nBased on uncorrected input reads:\r\n```\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_64-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n17.02e6\t400495\t97124\t400495\t500\t500\t929\t1797\t3307\t2389\t40026\t576.5e6\ttt_16D1C3L12__abyss_64-1.fa\r\n9269779\t400495\t97124\t400495\t500\t500\t929\t1797\t3307\t2389\t40026\t576.5e6\ttt_16D1C3L12__abyss_64-2.fa\r\n8970038\t386281\t91280\t386281\t500\t500\t1008\t1971\t3663\t2649\t48544\t597e6\ttt_16D1C3L12__abyss_64-3.fa\r\n3036\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_64-4.fa\r\n5148\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_64-5.fa\r\n8178432\t310669\t63719\t310669\t500\t500\t1495\t2882\t5772\t4006\t59732\t633.7e6\ttt_16D1C3L12__abyss_64-6.fa\r\n150\t23\t9\t23\t519\t519\t675\t918\t1344\t1079\t2400\t20808\ttt_16D1C3L12__abyss_64-7.fa\r\n8066430\t203300\t7924\t203300\t500\t500\t1783\t15776\t59085\t34309\t290392\t632.8e6\ttt_16D1C3L12__abyss_64-8.fa\r\n\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_80-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n13.12e6\t446607\t100548\t356890\t643\t500\t968\t2003\t3865\t2719\t40026\t684.4e6\ttt_16D1C3L12__abyss_80-1.fa\r\n7098314\t446607\t100548\t356890\t643\t500\t968\t2003\t3865\t2719\t40026\t684.4e6\ttt_16D1C3L12__abyss_80-2.fa\r\n6781726\t423529\t92333\t302719\t757\t500\t1071\t2247\t4374\t3073\t48480\t707.5e6\ttt_16D1C3L12__abyss_80-3.fa\r\n4693\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_80-4.fa\r\n6110\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_80-5.fa\r\n5820652\t308849\t55950\t157302\t1610\t500\t1887\t3813\t7894\t5337\t59232\t755e6\ttt_16D1C3L12__abyss_80-6.fa\r\n308\t99\t34\t99\t501\t501\t707\t977\t1455\t1144\t2825\t94745\ttt_16D1C3L12__abyss_80-7.fa\r\n5696462\t191129\t5702\t43046\t1934\t500\t2941\t28235\t91696\t52410\t565062\t753.4e6\ttt_16D1C3L12__abyss_80-8.fa\r\n\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_96-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n10.67e6\t480137\t100288\t266417\t964\t500\t1006\t2224\t4498\t3088\t40026\t779.6e6\ttt_16D1C3L12__abyss_96-1.fa\r\n5767063\t480137\t100288\t266417\t964\t500\t1006\t2224\t4498\t3088\t40026\t779.6e6\ttt_16D1C3L12__abyss_96-2.fa\r\n5440940\t445527\t89508\t221230\t1188\t500\t1142\t2565\t5220\t3560\t57744\t802.7e6\ttt_16D1C3L12__abyss_96-3.fa\r\n6539\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_96-4.fa\r\n6512\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_96-5.fa\r\n4474821\t304908\t49734\t107774\t2642\t500\t2204\t4824\t10075\t6702\t61751\t854.7e6\ttt_16D1C3L12__abyss_96-6.fa\r\n515\t208\t68\t208\t501\t501\t722\t1144\t1782\t1295\t3032\t218062\ttt_16D1C3L12__abyss_96-7.fa\r\n4344599\t184154\t4671\t14726\t10390\t500\t5460\t40717\t121439\t70922\t851373\t852.6e6\ttt_16D1C3L12__abyss_96-8.fa\r\n\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_112-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n8861505\t509781\t99296\t212832\t1294\t500\t1031\t2434\t5108\t3438\t45517\t864.1e6\ttt_16D1C3L12__abyss_112-1.fa\r\n4792535\t509781\t99296\t212832\t1294\t500\t1031\t2434\t5108\t3438\t45517\t864.1e6\ttt_16D1C3L12__abyss_112-2.fa\r\n4462563\t461389\t85975\t173733\t1635\t500\t1207\t2887\t6078\t4039\t49970\t884.7e6\ttt_16D1C3L12__abyss_112-3.fa\r\n8534\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_112-4.fa\r\n6145\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_112-5.fa\r\n3561326\t305368\t45960\t82288\t3652\t500\t2455\t5703\t11978\t7851\t79396\t936.6e6\ttt_16D1C3L12__abyss_112-6.fa\r\n663\t311\t99\t311\t500\t500\t798\t1193\t1927\t1362\t3365\t340199\ttt_16D1C3L12__abyss_112-7.fa\r\n3427972\t184013\t4178\t9292\t21190\t500\t7950\t50996\t146429\t85078\t664640\t934.1e6\ttt_16D1C3L12__abyss_112-8.fa\r\n\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_128-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n7603647\t540270\t98987\t178531\t1617\t500\t1042\t2609\t5644\t3732\t45325\t940.8e6\ttt_16D1C3L12__abyss_128-1.fa\r\n4164814\t540270\t98987\t178531\t1617\t500\t1042\t2609\t5644\t3732\t45325\t940.8e6\ttt_16D1C3L12__abyss_128-2.fa\r\n3835048\t475698\t83169\t143642\t2074\t500\t1260\t3192\t6791\t4462\t56412\t956e6\ttt_16D1C3L12__abyss_128-3.fa\r\n11372\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_128-4.fa\r\n5300\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_128-5.fa\r\n3011140\t310186\t43902\t67976\t4608\t500\t2643\t6415\t13450\t8757\t104425\t1.006e9\ttt_16D1C3L12__abyss_128-6.fa\r\n772\t429\t133\t429\t500\t500\t786\t1261\t2004\t1414\t3662\t482664\ttt_16D1C3L12__abyss_128-7.fa\r\n2876731\t189115\t4029\t7223\t30658\t500\t9361\t56647\t164847\t94949\t793681\t1.003e9\ttt_16D1C3L12__abyss_128-8.fa\r\n```\r\n\r\n\r\n@sjackman commented on Feb 13\r\n```\r\nn\tn:500\tL50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n8066430\t203300\t7924\t500\t1783\t15776\t59085\t34309\t290392\t632.8e6\ttt_16D1C3L12__abyss_64-8.fa\r\n2876731\t189115\t4029\t500\t9361\t56647\t164847\t94949\t793681\t1.003e9\ttt_16D1C3L12-8.fa\r\n```\r\n\r\n`k=128` looks like the better of these two assemblies to me. I'd suggest trying larger values of k. Note that the assembly size differs quite a bit between the two assemblies. That's something to keep in mind when interpreting these numbers. I'd suggest adding a `-G` parameter to `abyss-fac` that is your best estimate of the genome size to calculate `NG50`, which is generally better for comparing the contiguity of two assemblies. Note that `abyss-fac` can accept multiple FASTA files on its command line.\r\n\r\n\r\n@mmokrejs commented on Feb 13\r\n```\r\n$ abyss-fac -G 1267403131 ./abyss_k64/tt_16D1C3L12__abyss_64-scaffolds.fa ./abyss_k80/tt_16D1C3L12__abyss_80-scaffolds.fa ./abyss_k96/tt_16D1C3L12__abyss_96-scaffolds.fa tt_16D1C3L12__abyss_128-scaffolds.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n8066430\t203300\t7924\t203300\t500\t500\t1783\t15776\t59085\t34309\t290392\t632.8e6\t./abyss_k64/tt_16D1C3L12__abyss_64-scaffolds.fa\r\n5696462\t191129\t5702\t43046\t1934\t500\t2941\t28235\t91696\t52410\t565062\t753.4e6\t./abyss_k80/tt_16D1C3L12__abyss_80-scaffolds.fa\r\n4344599\t184154\t4671\t14726\t10390\t500\t5460\t40717\t121439\t70922\t851373\t852.6e6\t./abyss_k96/tt_16D1C3L12__abyss_96-scaffolds.fa\r\n2876731\t189115\t4029\t7223\t30658\t500\t9361\t56647\t164847\t94949\t793681\t1.003e9\t./abyss_k128/tt_16D1C3L12__abyss_128-scaffolds.fa\r\n```\r\n\r\nI am glad that now you agree the `k128` result is better than k64. Pity I confused you with the output from `stats.sh`. The `k112` and `k156` computations are ongoing, the `k156` will take 2 extra days to those about 35hrs for computation due to cluster maintenance.\r\n\r\n\r\n@sjackman commented on Feb 13\r\nYou could try some ABySS 2 bloom-filter dBG assemblies in the mean time if you liked, which require about a tenth of the RAM. Perhaps that'd be easier for you to get scheduled on your cluster.\r\n\r\n\r\n@mmokrejs commented on Mar 19\r\nSo, finally I have the assemblies with k160 and k192. I updated the tables above to reflect the new values.\r\n\r\n\r\n@sjackman commented on Mar 19\r\n`k=160` has the peak `NG50`. `k=128` has the peak `E-size` and `N50` and `largest` scaffold. Either of those look assemblies looks good to me.\r\n\r\n\r\n@mmokrejs commented on Mar 20\r\nI had the impression I shall try `k144` too. The input for `k160` and `k192` were error-corrected reads, for assemblies `<=k128` I used uncorrected reads. I will use the error-corrected reads for `k144` again. It decreases the memory usage and increases the average coverage by a tiny little but maybe still helps overall the assembly.\r\n\r\nBased on uncorrected input reads:\r\n```\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_128-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n7603647\t540270\t98987\t178531\t1617\t500\t1042\t2609\t5644\t3732\t45325\t940.8e6\ttt_16D1C3L12__abyss_128-1.fa\r\n4164814\t540270\t98987\t178531\t1617\t500\t1042\t2609\t5644\t3732\t45325\t940.8e6\ttt_16D1C3L12__abyss_128-2.fa\r\n3835048\t475698\t83169\t143642\t2074\t500\t1260\t3192\t6791\t4462\t56412\t956e6\ttt_16D1C3L12__abyss_128-3.fa\r\n11372\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_128-4.fa\r\n5300\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_128-5.fa\r\n3011140\t310186\t43902\t67976\t4608\t500\t2643\t6415\t13450\t8757\t104425\t1.006e9\ttt_16D1C3L12__abyss_128-6.fa\r\n772\t429\t133\t429\t500\t500\t786\t1261\t2004\t1414\t3662\t482664\ttt_16D1C3L12__abyss_128-7.fa\r\n2876731\t189115\t4029\t7223\t30658\t500\t9361\t56647\t164847\t94949\t793681\t1.003e9\ttt_16D1C3L12__abyss_128-8.fa\r\n```\r\n\r\nBased on error-corrected input reads (using tadpole.sh and k=64):\r\n```\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_128-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n7278490\t540152\t103434\t200909\t1417\t500\t1008\t2430\t5164\t3450\t42126\t905.4e6\ttt_16D1C3L12__abyss_128-1.fa\r\n3606504\t540152\t103434\t200909\t1417\t500\t1008\t2430\t5164\t3450\t42126\t905.4e6\ttt_16D1C3L12__abyss_128-2.fa\r\n3282085\t479704\t87706\t162732\t1801\t500\t1203\t2936\t6202\t4112\t57779\t921.7e6\ttt_16D1C3L12__abyss_128-3.fa\r\n8089\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_128-4.fa\r\n5252\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_128-5.fa\r\n2524721\t320183\t47968\t79411\t3892\t500\t2439\t5684\t11904\t7801\t84004\t972.2e6\ttt_16D1C3L12__abyss_128-6.fa\r\n1093\t563\t182\t563\t503\t503\t784\t1227\t1854\t1371\t3813\t624124\ttt_16D1C3L12__abyss_128-7.fa\r\n2383961\t195687\t4463\t8913\t23272\t500\t7332\t48678\t144350\t83684\t910150\t968.8e6\ttt_16D1C3L12__abyss_128-8.fa\r\n\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_144-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n6286575\t575941\t104590\t176213\t1671\t500\t995\t2539\t5531\t3653\t42158\t972.9e6\ttt_16D1C3L12__abyss_144-1.fa\r\n3128233\t575941\t104590\t176213\t1671\t500\t995\t2539\t5531\t3653\t42158\t972.9e6\ttt_16D1C3L12__abyss_144-2.fa\r\n2809379\t496455\t86074\t141270\t2144\t500\t1227\t3151\t6754\t4426\t56465\t980.4e6\ttt_16D1C3L12__abyss_144-3.fa\r\n10335\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_144-4.fa\r\n4403\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_144-5.fa\r\n2130364\t332146\t47269\t70164\t4540\t500\t2517\t6090\t12741\t8324\t109348\t1.027e9\ttt_16D1C3L12__abyss_144-6.fa\r\n1223\t658\t201\t658\t501\t501\t783\t1263\t1935\t1431\t3985\t736598\ttt_16D1C3L12__abyss_144-7.fa\r\n1988503\t207153\t4441\t7594\t29632\t500\t7711\t51296\t153330\t89693\t730307\t1.023e9\ttt_16D1C3L12__abyss_144-8.fa\r\n\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_160-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n5378996\t618241\t106811\t157364\t1918\t500\t965\t2616\t5854\t3810\t42190\t1.041e9\ttt_16D1C3L12__abyss_160-1.fa\r\n2722381\t618241\t106811\t157364\t1918\t500\t965\t2616\t5854\t3810\t42190\t1.041e9\ttt_16D1C3L12__abyss_160-2.fa\r\n2411515\t516769\t85211\t125222\t2482\t500\t1231\t3345\t7223\t4688\t57811\t1.037e9\ttt_16D1C3L12__abyss_160-3.fa\r\n12737\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_160-4.fa\r\n3101\t1\t1\t1\t501\t501\t501\t501\t501\t501\t501\t501\ttt_16D1C3L12__abyss_160-5.fa\r\n1807627\t348775\t47265\t63826\t5107\t500\t2543\t6370\t13388\t8717\t82251\t1.078e9\ttt_16D1C3L12__abyss_160-6.fa\r\n1263\t717\t215\t717\t502\t502\t756\t1240\t1970\t1481\t7966\t794484\ttt_16D1C3L12__abyss_160-7.fa\r\n1665496\t223249\t4585\t6863\t34706\t500\t7480\t51976\t156903\t91040\t724617\t1.074e9\ttt_16D1C3L12__abyss_160-8.fa\r\n\r\n$ abyss-fac -G 1267403131 tt_16D1C3L12__abyss_192-?.fa\r\nn\tn:500\tL50\tLG50\tNG50\tmin\tN80\tN50\tN20\tE-size\tmax\tsum\tname\r\n3646761\t728516\t121430\t143397\t2194\t500\t874\t2503\t5844\t3754\t51286\t1.165e9\ttt_16D1C3L12__abyss_192-1.fa\r\n2014683\t728516\t121430\t143397\t2194\t500\t874\t2503\t5844\t3754\t51286\t1.165e9\ttt_16D1C3L12__abyss_192-2.fa\r\n1731479\t596980\t93440\t114146\t2829\t500\t1112\t3294\t7312\t4686\t60880\t1.141e9\ttt_16D1C3L12__abyss_192-3.fa\r\n33949\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\t0\ttt_16D1C3L12__abyss_192-4.fa\r\n1201\t1\t1\t1\t507\t507\t507\t507\t507\t507\t507\t507\ttt_16D1C3L12__abyss_192-5.fa\r\n1361846\t455052\t61113\t71142\t4723\t500\t1948\t5287\t11270\t7278\t98989\t1.167e9\ttt_16D1C3L12__abyss_192-6.fa\r\n767\t579\t167\t579\t500\t500\t749\t1314\t2091\t1500\t6596\t650527\ttt_16D1C3L12__abyss_192-7.fa\r\n1213455\t319826\t6263\t7774\t30914\t500\t2610\t38588\t129848\t74019\t751657\t1.163e9\ttt_16D1C3L12__abyss_192-8.fa\r\n```\r\n\r\n@sjackman would you please comment on the k144 and k160 assemblies. I think k144 is the best from k128-k192 range (has very high actually assmbled genome size and still very high max scaffold, `L50` is even better than in `k128`, NG50 is somewhat in between) . Provided the gap closing steps are so time consuming, isn't it however better approach to opt for `k160` assembly? It provides almost complete genome (unless the actually assembled genome size is inflated due to redundant contigs) and is maybe a better base for future work.\r\n\r\nI am running \r\n\r\n`java -Xmx2800G -jar pilon-1.22.jar --output abyss_ecc_k128_ecc_N10_pilon --diploid --changes --vcf --fix all,amb  --threads 104  ...`\r\n\r\nbut it will last for about 1200wallclock hours. As we discussed elsewhere, abyss' Sealer runs multithreaded in the very beginning but then switches to a single-threaded job. And moreover, I have the impression it would just make a perfect consensus of my sequences inside \"gaps\"  whereas pilon scans the BAM files and works with the spanning read pairs (so it could pick the right set of two alleles). Unfortunately, it works only with the proper pairs and way too many read-pairs I obtained from BBmap have \"wrong\" orientation or insert size (note the mix in Nextera mate-pair libs) so it works with much less of reads. What is your strategy? I shall try the BAM's created by `abyss-map` too if it cares to set the proper read flags. BWA doesn't I think so that is why I did not try.\r\n\r\n\r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/bcgsc/abyss/issues/comments/385194141","html_url":"https://github.com/bcgsc/abyss/issues/220#issuecomment-385194141","issue_url":"https://api.github.com/repos/bcgsc/abyss/issues/220","id":385194141,"user":{"login":"mmokrejs","id":3297852,"avatar_url":"https://avatars1.githubusercontent.com/u/3297852?v=4","gravatar_id":"","url":"https://api.github.com/users/mmokrejs","html_url":"https://github.com/mmokrejs","followers_url":"https://api.github.com/users/mmokrejs/followers","following_url":"https://api.github.com/users/mmokrejs/following{/other_user}","gists_url":"https://api.github.com/users/mmokrejs/gists{/gist_id}","starred_url":"https://api.github.com/users/mmokrejs/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mmokrejs/subscriptions","organizations_url":"https://api.github.com/users/mmokrejs/orgs","repos_url":"https://api.github.com/users/mmokrejs/repos","events_url":"https://api.github.com/users/mmokrejs/events{/privacy}","received_events_url":"https://api.github.com/users/mmokrejs/received_events","type":"User","site_admin":false},"created_at":"2018-04-28T17:52:52Z","updated_at":"2018-04-28T17:52:52Z","author_association":"CONTRIBUTOR","body":"But Pilon seems to require 1.2TB at least for my data. I ran out of memory a few times when I tested with no extra `java -Xmx` flag.\r\n\r\nMaybe pilon is just unpacking all the time my BAM files to realize there is not enough coverage? But working with SAM files would be probably much faster. i again forgot the difference in their indexing approaches."}}
{"actor":{"display_login":"mmokrejs"}}