text: user9931: I'm pretty new to Unix and bioinformatics and having a hard time accomplishing the following. I have one FASTA file with sequences and headers, and one OTU ID mapping file (.txt) with OTU IDs and their sequences. The sequence header in the .fasta file partially matches the sequence listed in the OTU ID file. What I want to do is replace the current headers in the FASTA file with just the OTU IDs from the .txt file. The files look like something like this:
  OTU_ID.txt
  denovo123 U1019.1010.2020_20384
  denovo453 U1472.2992.2020_20443
  denovo763 U1928.1898.2020_20343
  
  .FASTA
  >U1019.1010.2020_20384 0000-AHDJ-38347_223
  GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  >U1928.1898.2020_20343 0000-AHDJ-24247_242
  GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  
  and my desired output is that the .fasta file would look like this:
  >denovo123
  GAAATACCGAGAGAGAAATACCGAGAGAGAAAGAAATACCGAGAGAGAAATACCGAGAAATACCGAGAGAGAAATACCGAGAGA
  >denovo763
  GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  
  Is it possible to do this in the command line?
  
  user3360: You can do it with bash quiet easy. First create a backup of your fasta file with
  cp otu.fasta otu_backup.fasta
  
  Then you can run:
  while IFS=" " read -r otu fastaId rest; 
  do
  sed -E -i "s/>$fastaId .+/>$otu/" otu.fasta; done < ids.txt;
  done < OTU_ID.txt
  
  What does the code?
  With IFS=" " you declare a whitespace as field seperator and by reading each line of your OTU_ID.txt, you have the two variables otu and fastaId which hold the otu name and the id of the Fasta Header.
  Within the while loop, sed is looking each loop for the corresponding id within a RegEx and substitute it with the corresponding otu name. By using the -i option, you overwrite the existing file. This is why you need to create a backup.
  
  user4282: You can put you fasta file in a single line per sequence:
  perl -pe 'if(/\>/){s/\n/\t/; s/>//; s/\n//}' file.fasta > file.tsv
  
  This line will create the next output:
  U1019.1010.2020_20384 0000-AHDJ-38347_223   GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  U1928.1898.2020_20343 0000-AHDJ-24247_242   GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  
  Then with awk you can match columns separated with tab:
  awk -F' ' 'FNR==NR{a[$2]=$1;next} ($1 in a) {print ">"a[$1]"\n"$3}' otu_id.txt file.tsv
  
  You will get this output:
  >denovo123
  GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  >denovo763
  AAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  
  You can add $1 to check if you are getting the right id.
  awk -F' ' 'FNR==NR{a[$2]=$1;next} ($1 in a) {print ">"a[$1], $1, "\n"$3}' otu_id.txt file.tsv
  
  If everything is alright you can redirect to a new file:
  awk -F' ' 'FNR==NR{a[$2]=$1;next} ($1 in a) {print ">"a[$1]"\n"$3}' otu_id.txt file.tsv > file_ids.fasta
  
  How it works, well awk implicitly loops through each file, one line at a time.
  FNR==NR{a[$2]=$1;next}
  NR is the number of lines that awk has read so far and FNR is the number of lines that awk has read from the current file. Thus, if FNR==NR, we are still reading the first-named file: otu_id.txt. For every line in file.tsv, we assign a[$2]=$1.
  Here, a is an associative array, and a[$2]=$1 means saving file.tsv first column, denoted $1, as a value in array a using file.tsv second column, $2, as the key.
  next tells awk to skip the rest of the commands and start over with the next line.
  ($1 in a) {print ">"a[$1]"\n"$3}
  This means that we are reading the second file: otu_id.tsv. If match the first field of the line in file.tsv, as determined by the contents of array a, it will print out a line with the values of fields a[$1] and $3 separated with a new line character.
  
original_text: 9931: <p>I'm pretty new to Unix and bioinformatics and having a hard time accomplishing the following. I have one FASTA file with sequences and headers, and one OTU ID mapping file (.txt) with OTU IDs and their sequences. The sequence header in the .fasta file partially matches the sequence listed in the OTU ID file. What I want to do is replace the current headers in the FASTA file with just the OTU IDs from the .txt file. The files look like something like this:</p>
  <p>OTU_ID.txt</p>
  <pre><code>denovo123 U1019.1010.2020_20384
  denovo453 U1472.2992.2020_20443
  denovo763 U1928.1898.2020_20343
  </code></pre>
  <p>.FASTA</p>
  <pre><code>&gt;U1019.1010.2020_20384 0000-AHDJ-38347_223
  GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  &gt;U1928.1898.2020_20343 0000-AHDJ-24247_242
  GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  </code></pre>
  <p>and my desired output is that the .fasta file would look like this:</p>
  <pre><code>&gt;denovo123
  GAAATACCGAGAGAGAAATACCGAGAGAGAAAGAAATACCGAGAGAGAAATACCGAGAAATACCGAGAGAGAAATACCGAGAGA
  &gt;denovo763
  GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  </code></pre>
  <p>Is it possible to do this in the command line?</p>
   3360: <p>You can do it with bash quiet easy. First create a backup of your fasta file with</p>
  <pre><code>cp otu.fasta otu_backup.fasta
  </code></pre>
  <p>Then you can run:</p>
  <pre><code>while IFS=&quot; &quot; read -r otu fastaId rest; 
  do
  sed -E -i &quot;s/&gt;$fastaId .+/&gt;$otu/&quot; otu.fasta; done &lt; ids.txt;
  done &lt; OTU_ID.txt
  </code></pre>
  <p>What does the code?</p>
  <p>With <code>IFS=&quot; &quot;</code> you declare a whitespace as field seperator and by reading each line of your <code>OTU_ID.txt</code>, you have the two variables <code>otu</code> and <code>fastaId</code> which hold the otu name and the id of the Fasta Header.</p>
  <p>Within the <code>while</code> loop, sed is looking each loop for the corresponding id within a RegEx and substitute it with the corresponding otu name. By using the <code>-i</code> option, you overwrite the existing file. This is why you need to create a backup.</p>
   4282: <p>You can put you fasta file in a single line per sequence:</p>
  <pre><code>perl -pe 'if(/\&gt;/){s/\n/\t/; s/&gt;//; s/\n//}' file.fasta &gt; file.tsv
  </code></pre>
  <p>This line will create the next output:</p>
  <pre><code>U1019.1010.2020_20384 0000-AHDJ-38347_223   GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  U1928.1898.2020_20343 0000-AHDJ-24247_242   GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  </code></pre>
  <p>Then with awk you can match columns separated with tab:</p>
  <pre><code>awk -F' ' 'FNR==NR{a[<span class="math-container">$2]=$</span>1;next} (<span class="math-container">$1 in a) {print "&gt;"a[$1]"\n"$</span>3}' otu_id.txt file.tsv
  </code></pre>
  <p>You will get this output:</p>
  <pre><code>&gt;denovo123
  GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  &gt;denovo763
  AAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA
  </code></pre>
  <p>You can add <strong><code>$1</code></strong> to check if you are getting the right id.</p>
  <pre><code>awk -F' ' 'FNR==NR{a[<span class="math-container">$2]=$</span>1;next} (<span class="math-container">$1 in a) {print "&gt;"a[$1], $1, "\n"$</span>3}' otu_id.txt file.tsv
  </code></pre>
  <p>If everything is alright you can redirect to a new file:</p>
  <pre><code>awk -F' ' 'FNR==NR{a[<span class="math-container">$2]=$</span>1;next} (<span class="math-container">$1 in a) {print "&gt;"a[$1]"\n"$</span>3}' otu_id.txt file.tsv &gt; file_ids.fasta
  </code></pre>
  <p>How it works, well <code>awk</code> implicitly loops through each file, one line at a time.</p>
  <p><code>FNR==NR{a[$2]=$1;next}</code></p>
  <p><code>NR</code> is the number of lines that awk has read so far and <code>FNR</code> is the number of lines that <code>awk</code> has read from the current file. Thus, if <code>FNR==NR</code>, we are still reading the first-named file: otu_id.txt. For every line in file.tsv, we assign <code>a[$2]=$1</code>.</p>
  <p>Here, <code>a</code> is an associative array, and <code>a[$2]=$1</code> means saving file.tsv first column, denoted <code>$1</code>, as <code>a</code> value in array a using file.tsv second column, <code>$2</code>, as the key.</p>
  <p><code>next</code> tells <code>awk</code> to skip the rest of the commands and start over with the next line.</p>
  <p><code>($1 in a) {print &quot;&gt;&quot;a[$1]&quot;\n&quot;$3}</code></p>
  <p>This means that we are reading the second file: otu_id.tsv. If match the first field of the line in file.tsv, as determined by the contents of array <code>a</code>, it will print out a line with the values of fields <code>a[$1]</code> and <code>$3</code> separated with a new line character.</p>
  