{"action":"created","issue":{"url":"https://api.github.com/repos/benjjneb/dada2/issues/100","repository_url":"https://api.github.com/repos/benjjneb/dada2","labels_url":"https://api.github.com/repos/benjjneb/dada2/issues/100/labels{/name}","comments_url":"https://api.github.com/repos/benjjneb/dada2/issues/100/comments","events_url":"https://api.github.com/repos/benjjneb/dada2/issues/100/events","html_url":"https://github.com/benjjneb/dada2/issues/100","id":169027389,"number":100,"title":"Subsampling a fraction of reads across all samples for error rate estimation","user":{"login":"elsherbini","id":2695357,"avatar_url":"https://avatars.githubusercontent.com/u/2695357?v=3","gravatar_id":"","url":"https://api.github.com/users/elsherbini","html_url":"https://github.com/elsherbini","followers_url":"https://api.github.com/users/elsherbini/followers","following_url":"https://api.github.com/users/elsherbini/following{/other_user}","gists_url":"https://api.github.com/users/elsherbini/gists{/gist_id}","starred_url":"https://api.github.com/users/elsherbini/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/elsherbini/subscriptions","organizations_url":"https://api.github.com/users/elsherbini/orgs","repos_url":"https://api.github.com/users/elsherbini/repos","events_url":"https://api.github.com/users/elsherbini/events{/privacy}","received_events_url":"https://api.github.com/users/elsherbini/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2016-08-03T01:33:00Z","updated_at":"2016-08-04T20:41:37Z","closed_at":null,"body":"Mentioned in another issue [here](https://github.com/benjjneb/dada2/issues/41#issuecomment-236462931), it seems the best practice way to estimate the error rates on a subset of the data is to pick some number of samples that should add up to some fraction of the total reads.\r\n\r\nI'm wondering if there is a convenient way to get a random sample of reads across all samples. @joey711 mentioned [here](https://github.com/benjjneb/dada2/issues/41#issuecomment-237102330) that it might be important to be careful in doing this, so as to get a representative sample.\r\n\r\nMy naive idea would be to simply cat all the sample fastqs together and then pick at random, maybe using GNU utils `shuf` command. If you want to make sure you sample matched pairs of reads, there might be slightly more code involved, but it'd be the same principle."},"comment":{"url":"https://api.github.com/repos/benjjneb/dada2/issues/comments/237677027","html_url":"https://github.com/benjjneb/dada2/issues/100#issuecomment-237677027","issue_url":"https://api.github.com/repos/benjjneb/dada2/issues/100","id":237677027,"user":{"login":"joey711","id":841437,"avatar_url":"https://avatars.githubusercontent.com/u/841437?v=3","gravatar_id":"","url":"https://api.github.com/users/joey711","html_url":"https://github.com/joey711","followers_url":"https://api.github.com/users/joey711/followers","following_url":"https://api.github.com/users/joey711/following{/other_user}","gists_url":"https://api.github.com/users/joey711/gists{/gist_id}","starred_url":"https://api.github.com/users/joey711/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joey711/subscriptions","organizations_url":"https://api.github.com/users/joey711/orgs","repos_url":"https://api.github.com/users/joey711/repos","events_url":"https://api.github.com/users/joey711/events{/privacy}","received_events_url":"https://api.github.com/users/joey711/received_events","type":"User","site_admin":false},"created_at":"2016-08-04T20:41:37Z","updated_at":"2016-08-04T20:41:37Z","body":"Your intuition is definitely off. Not that this is very intuitive. It's not. So thanks for being honest. I'm guessing this post will help other users as well. Perhaps reread my explanation, or even better yet, read Ben's concise description of the problem more carefully. \r\n\r\n**Point (1)**. Your point about jackpotting is a good one, though. Ben and I are not arguing for using only a pointlessly-small number of samples that leave you very worried about effects like outliers. There's no point. You should be able to convince yourself (by comparing with sample-wise error rates) that the rates are converging around representative values for that seq run. If you have reason to believe a sample has a problem, or you see later in the data analysis that it exhibited unreliable, artifact-suggestive results, you probably want to make sure that you didn't use it as one of a small number of samples for the error rates estimation. Provided you didn't get unlucky and pick terrible samples from an otherwise good sequencing run, it shouldn't matter which samples you pick. That's the part that might not be immediately intuitive. The error rates should be consistent across samples regardless of their origin, provided they are in the same sequencing run and there weren't other major confounders that you'd expect to generate different types of errors in an amplicon-sequencing experiment.\r\n\r\n**Point (2)**. Sample size variation is not an issue here on its own. In some cases a very small sample size is indicative of a failed sample/prep/seq, and its data should be removed from the dataset. I can see I wasn't clear enough in my description, partly because I was emphasizing that you should stick with the sample-wise subsetting. However, in the example you're describing, the 100-read sample would not be useful for estimating error rates on its own (for most microbiomes), while the 1-million read sample would probably do a very good job by itself in returning a reliable error-rate estimation that works for the rest of the reads in that run. In practice, 1-million read sample might be a sample that was run by itself. But to finish the example anyway, the 1-million read sample might have its error-rates well estimated by a uniform downsampling of 20,000 reads. The actual minimum read depth for convergence depends on the underlying data, but 20,000 is not a bad default value for MiSeq-length V4 data from fecal data, for example. You can always try a few different downsampled depths from that large file, and check at which level the error-rates converge. For most microbiomes they would have converged by that size. This is also a way of controlling the computation time (and peak memory? @benjjneb ?) required even when you have very large read-depths per sample.\r\n\r\n\r\nRe-summary, hopefully more clear:\r\n\r\nThe point is that too-shallow a sampling from each file (sample/specimen) is clearly a problem, and one that can be avoided by using enough of the reads from each sample included in the error-rate estimation step. How many reads will depend on the data, though, and the typical depth per sample used in current experimental designs is already enough in most cases. Hence the utility of doing it sample-wise. It allows you to avoid (most of) the worrying about choosing/evaluating this per-sample-size parameter.\r\n\r\n\r\n"}}
{"actor":{"display_login":"joey711"}}