text: user96: With tabix one can index a VCF file for rapid variant retrieval based on genomic position. I'm wondering if there are any tools that will index a VCF file to allow rapid retrieval using rsIDs and/or other metadata? I'm aware of awk/grep/vcftools one-liners for this purpose, but I'd like to avoid scanning a huge VCF each time I need to retrieve the coordinates of a new batch of rsIDs.
  
  user492: Take a look at grabix. Using grabix you could implement a binary search on a VCF sorted by rsID and compressed using bgzip. 
  
  user134: I am not aware of any widespread index formats for query-by-QNAME for BAM or query-by-ID for VCF. I thought there were historical samtools{-devel,-help} threads to the effect of “in principle one could index a named-sorted BAM file, but there's insufficient demand so no-one's ever implemented it”, but I can't find any just now.
  Moreover note that VCF files are by definition sorted by genomic position, so such an index would just be a hash table of rsID to file offset or genomic position.
  If you are using dbSNP, probably you have a dbSNP database or API that you can use to query by rsID and get genomic position back. So you can retrieve your VCF contents by rsID in two stages: do that dbSNP query, and use its results to find the variants in your VCF file by position.
  
  user15096: high-perf-bio.
  I present my high-perf-bio project, which, for the most part, solves the problem of fast extraction of anything values from VCF, BED and unformatted data.
  Preparation.
  1/ Run the automatic downloading of the toolkit and installation of the dependencies.
  wget https://github.com/PlatonB/high-perf-bio/archive/refs/heads/master.zip && unzip -o master.zip && rm master.zip && cd high-perf-bio-master && bash __install_3rd_party.sh
  
  2/ Reboot.
  3/ cd path/to/high-perf-bio-master
  Create DB with rsIDs.
  python create.py -S path/to/gzipped_tables_dir -i rsIDs_field_path
  
  Search rsIDs set in DB.
  python annotate.py -S path/to/annotated_gzipped_tables_dir -D db_name -T path/to/trg_dir #-c rsIDs_column_num -f rsIDs_field_path
  
  P.S.1. Detailed help for each high-perf-bio component: python tool_name.py -h
  P.S.2. For VCF and BED you can specify the minimum of arguments.
  P.S.3. Readme in English is in the early stages of writing. If you have any questions, write to Issues.
  
original_text: 96: <p>With tabix one can index a VCF file for rapid variant retrieval based on genomic position. I'm wondering if there are any tools that will index a VCF file to allow rapid retrieval using rsIDs and/or other metadata? I'm aware of awk/grep/vcftools one-liners for this purpose, but I'd like to avoid scanning a huge VCF each time I need to retrieve the coordinates of a new batch of rsIDs.</p>
   492: <p>Take a look at <a href="https://github.com/arq5x/grabix" rel="nofollow noreferrer"><code>grabix</code></a>. Using <code>grabix</code> you could implement a binary search on a VCF sorted by rsID and compressed using <code>bgzip</code>. </p>
   134: <p>I am not aware of any widespread index formats for query-by-QNAME for BAM or query-by-ID for VCF. I thought there were historical <em>samtools{-devel,-help}</em> threads to the effect of “in principle one could index a named-sorted BAM file, but there's insufficient demand so no-one's ever implemented it”, but I can't find any just now.</p>
  
  <p>Moreover note that VCF files are by definition sorted by genomic position, so such an index would just be a hash table of rsID to file offset or genomic position.</p>
  
  <p>If you are using dbSNP, probably you have a dbSNP database or API that you can use to query by rsID and get genomic position back. So you can retrieve your VCF contents by rsID in two stages: do that dbSNP query, and use its results to find the variants in your VCF file by position.</p>
   15096: <p><strong>high-perf-bio.</strong></p>
  <p>I present my <a href="https://github.com/PlatonB/high-perf-bio" rel="nofollow noreferrer"><em>high-perf-bio</em></a> project, which, for the most part, solves the problem of fast extraction of anything values from VCF, BED and unformatted data.</p>
  <p><strong>Preparation.</strong></p>
  <p>1/ Run the automatic downloading of the toolkit and installation of the dependencies.</p>
  <pre><code>wget https://github.com/PlatonB/high-perf-bio/archive/refs/heads/master.zip &amp;&amp; unzip -o master.zip &amp;&amp; rm master.zip &amp;&amp; cd high-perf-bio-master &amp;&amp; bash __install_3rd_party.sh
  </code></pre>
  <p>2/ Reboot.</p>
  <p>3/ <code>cd path/to/high-perf-bio-master</code></p>
  <p><strong>Create DB with rsIDs.</strong></p>
  <pre><code>python create.py -S path/to/gzipped_tables_dir -i rsIDs_field_path
  </code></pre>
  <p><strong>Search rsIDs set in DB.</strong></p>
  <pre><code>python annotate.py -S path/to/annotated_gzipped_tables_dir -D db_name -T path/to/trg_dir #-c rsIDs_column_num -f rsIDs_field_path
  </code></pre>
  <p>P.S.1. Detailed help for each <em>high-perf-bio</em> component: <code>python tool_name.py -h</code></p>
  <p>P.S.2. For VCF and BED you can specify the minimum of arguments.</p>
  <p>P.S.3. Readme in English is in the early stages of writing. If you have any questions, write to <a href="https://github.com/PlatonB/high-perf-bio/issues" rel="nofollow noreferrer">Issues</a>.</p>
  