text: user8544: Multiple sources (see for instance this or this) discuss how genetic data will have scalability problems, given the huge file size of the human genome. The most straightforward encoding (see here) of the human genome requires about 700 Mb. 
  I came across this paper claiming to be able to store the human genome in about 4 Mb, having a reference genome and exploiting the fact that all human genomes are mostly equal. This paper is much older than the the other references discussing the scalability problems. Why is this technique not widely used? 
  
  user383: I saw this paper a few weeks back that claims to offer 4x more compression than typical formats for both fastqs and BAM/CRAM files.
  Genozip - A Universal Extensible Genomic Data Compressor
  https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btab102/6135077
  
  user73: Answer from @alex-reynolds, converted from comments:
  As with any compression scheme, if you have to store a reference genome, along with the 4 Mb "diff", is it really compressed?
  http://mattmahoney.net/dc/dce.html
  For sequencing data, you might also look at the CRAM format, which also uses diffs against a reference genome to store reduced datasets. Labs are switching from BAM to CRAM to manage storage of increased numbers of sequencing datasets, with associated changes to pipelines and toolkits to use this format. For performance reasons, BAM may be preferred for analysis, and CRAM for long-term storage.
  https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats
  https://www.ga4gh.org/news/cram-compression-for-genomics/
  
original_text: 8544: <p>Multiple sources (see for instance <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5958914/" rel="nofollow noreferrer">this</a> or <a href="https://spectrum.ieee.org/computing/software/the-desperate-quest-for-genomic-compression-algorithms" rel="nofollow noreferrer">this</a>) discuss how genetic data will have scalability problems, given the huge file size of the human genome. The most straightforward encoding (see <a href="https://stackoverflow.com/questions/8954571/how-much-storage-would-be-required-to-store-a-human-genome">here</a>) of the human genome requires about 700 Mb. </p>
  
  <p>I came across this <a href="https://academic.oup.com/bioinformatics/article/25/2/274/218156" rel="nofollow noreferrer">paper</a> claiming to be able to store the human genome in about 4 Mb, having a reference genome and exploiting the fact that all human genomes are mostly equal. This paper is much older than the the other references discussing the scalability problems. Why is this technique not widely used? </p>
   383: <p>I saw this paper a few weeks back that claims to offer 4x more compression than typical formats for both fastqs and BAM/CRAM files.</p>
  <p>Genozip - A Universal Extensible Genomic Data Compressor</p>
  <p><a href="https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btab102/6135077" rel="nofollow noreferrer">https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btab102/6135077</a></p>
   73: <p><em>Answer from @alex-reynolds, converted from comments:</em></p>
  <p>As with any compression scheme, if you have to store a reference genome, along with the 4 Mb &quot;diff&quot;, is it really compressed?</p>
  <p><a href="http://mattmahoney.net/dc/dce.html" rel="nofollow noreferrer">http://mattmahoney.net/dc/dce.html</a></p>
  <p>For sequencing data, you might also look at the CRAM format, which also uses diffs against a reference genome to store reduced datasets. Labs are switching from BAM to CRAM to manage storage of increased numbers of sequencing datasets, with associated changes to pipelines and toolkits to use this format. For performance reasons, BAM may be preferred for analysis, and CRAM for long-term storage.</p>
  <p><a href="https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats" rel="nofollow noreferrer">https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats</a></p>
  <p><a href="https://www.ga4gh.org/news/cram-compression-for-genomics/" rel="nofollow noreferrer">https://www.ga4gh.org/news/cram-compression-for-genomics/</a></p>
  