title: perl script to select best blast gene line from blast result
body: <p>I have huge file with blast output and I need to select the query ID, subject gi and the frame (basically the whole line) with lowest e-value omitting the duplicate lines (omitting all other lines with other higher e-values).  This is how the file looks like:</p>
  <pre><code># BLASTX 2.2.28+
  # 0 hits found
  # BLASTX 2.2.28+
  # Query: Tx6_c1_seq1
  # Database: /mnt/swissprot
  # Fields: query id, subject gi, subject title, subject length, gap opens, q. start, q. end, s. start, s. end, evalue, % subject coverage, % identity, query/sbjct frames
  # 24 hits found
  Tx6_c1_seq1 6439823 RecName: Full=E3 ubiquitin-protein ligase siah-1; AltName: Full=Seven in absentia homolog 1 434 1   9   173 224 282 1e-06   65  32.20   3/0
  Tx6_c1_seq1 577332  RecName: Full=Putative E3 ubiquitin-protein ligase SINAT1; AltName: Full=Seven in absentia homolog 1    305 1   9   179 111 171 3e-05   67  32.79   3/0
  Tx6_c1_seq1 3548505 RecName: Full=E3 ubiquitin-protein ligase siah-1; AltName: Full=Seven in absentia homolog 1 419 2   9   173 209 267 8e-05   65  32.20   3/0
  Tx6_c1_seq1 577547  RecName: Full=E3 ubiquitin-protein ligase siah2; AltName: Full=Seven in absentia homolog 2; AltName: Full=Xsiah-2   313 1   15  173 125 181 2e-04   62  29.82   3/0
  Tx6_c1_seq1 577417  RecName: Full=E3 ubiquitin-protein ligase Siah1; AltName: Full=Seven in absentia homolog 1; Short=Siah-1    282 1   15  173 96  152 3e-04   62  29.82   3/0
  Tx6_c1_seq1 577554  RecName: Full=E3 ubiquitin-protein ligase SINAT2; AltName: Full=Seven in absentia homolog 2 308 1   9   179 114 174 4e-04   67  31.15   3/0
  # BLASTX 2.2.28+
  # Query: Tx_11_c0_seq1
  # Database: /mnt/swissprot
  # Fields: query id, subject gi, subject title, subject length, gap opens, q. start, q. end, s. start, s. end, evalue, % subject coverage, % identity, query/sbjct frames
  # 1 hits found
  Tx_11_c0_seq1   977285  RecName: Full=120.7 kDa protein in NOF-FB transposable element  1056    15  957 28  147 455 8e-13   79  27.81   -2/0
  # BLASTX 2.2.28+
  # Query: Tx_11_c1_seq1
  </code></pre>
  <p>The expected output in this case should only be these two line as they are the ones with smallest e_value:</p>
  <pre><code>Tx6_c1_seq1 6439823 RecName: Full=E3 ubiquitin-protein ligase siah-1; AltName: Full=Seven in absentia homolog 1 434 1   9   173 224 282 1e-06   65  32.20   3/0
  Tx_11_c0_seq1   977285  RecName: Full=120.7 kDa protein in NOF-FB transposable element  1056
  </code></pre>
  <p>I have my code written, but does not seem to work. Could you guys please help me to resolve this problem. I would really appreciate your time and help. This is what I have so far:</p>
  <pre><code>#!/usr/bin/perl -w
  
  # Author:
  # 01/07/2014
  # This script removes duplicate records from a &quot;short&quot; format BLAST output file, and keeps only the &quot;best&quot; records  (sorts by smallest e-value and then biggest percent identity)
  # Usage: bestblast.pl &lt;input file&gt; &lt;output file&gt;
  
  #-----------------------------------------------------------------------------------------------------------------------------------------
  #Deal with passed parameters
  #-----------------------------------------------------------------------------------------------------------------------------------------
  #If no arguments are passed, show usage message and exit program.
  if ($#ARGV == -1) {
      usage(&quot;BLAST BEST 1.0 2014&quot;);
      exit;
  }
  
  #get the names of the input file (first argument passed) and output file (second argument passed)
  $in_file = $ARGV[0];
  $out_file = $ARGV[1];
  
  #Open the input file for reading, open the output file for writing.
  #If either are unsuccessful, print an error message and exit program.
  unless ( open(IN, &quot;$in_file&quot;) ) {
      usage(&quot;Got a bad input file: $in_file&quot;);
      exit;
  }
  unless ( open(OUT, &quot;&gt;$out_file&quot;) ) {
      usage(&quot;Got a bad output file: $out_file&quot;);
      exit;
  }
  
  #Everything looks good. Print the parameters we've found.
  print &quot;Parameters:\ninput file = $in_file\noutput file = $out_file\n\n&quot;;
  
  #-----------------------------------------------------------------------------------------------------------------------------------------
  #The main event
  #-----------------------------------------------------------------------------------------------------------------------------------------
  
  $counter = 0;
  $total_counter = 0;
  
  print &quot;De-duplicating File...\n&quot;;
  
  @in = &lt;IN&gt;;
  
  #Do stuff for each line of text in the input file.
  foreach $line (@in) {
      #if the line starts with a pound symbol, it is not real data, so skip this line.
      if ( $line =~ /^#/ ) {
       next;
       }
  
      #Count the total number of data lines in the file.
      $total_counter++;
  
      #The chomp commands removes any new line (and carriage return) characters from the end of the line.
      chomp($line);
  
      #Split up the tab delimited line, naming only the variables we are interested in (i.e. query id, subject gi, subject title, subject length, gap opens, q. start, q. end, s. start, s. end, evalue, % subject coverage, % identity, query/sbjct frames)
      ($query_id, $subject_gi, $subject_title, $subject_length, $gap_opens, $q_start, $q_end, $s_start, $s_end, $evalue, $subject_coverage, $identity, $query_sbjct_frames) = split(/\t/, $line);
  
      #check to see if the id label is already in the list of ids (called dedupe)
      #if its not there, add it.
      if ( $dedupe{$query_id} ) {
      #if it is, look at the old line to see if it is still &quot;better&quot; than the new one.
      ($query_id, $subject_gi, $subject_title, $subject_length, $gap_opens, $q_start, $q_end, $s_start, $s_end, $list_evalue, $subject_coverage, $list_identity, $query_sbjct_frames) = split(/\t/,$dedupe{$query_id});
  
      #if the new evalue is better than the old one, change the value of this id to the new line.
      #otherwise, if the new evalue is the same, and the percent_identity is better, change the value of this id to the new line.
      #otherwise, don't do anything (keep the old line).
      if ( $evalue &lt; $list_evalue ) {
          $dedupe{$query_id} = $line;
      }
      elsif ( $evalue == $list_evalue ) {
          if ( $identity &gt; $list_identity ) {
          $dedupe{$query_id} = $line;
          }
      }
      }
      else {
      $dedupe{$query_id} = $line;
      #count the number of non-duplicated lines we have.
      $counter++;
      }
  }
  print &quot;Total # records = $total_counter\nBest only # records = $counter\n&quot;;
  print &quot;Writing to output file...\n&quot;;
  
  #Print the final &quot;dedupe&quot; list to the new file (adding the new line back on the end).
  foreach $query_id (sort keys %dedupe) {
      print OUT &quot;$dedupe{$query_id}\n&quot;;
  }
  
  #Close the files.
  close(IN);
  close(OUT);
  print &quot;Done.\n&quot;;
  
  #-----------------------------------------------------------------------------------------------------------------------------------------
  #Subroutines
  #-----------------------------------------------------------------------------------------------------------------------------------------
  sub usage {
      my($message) = @_;
      print &quot;\n$message\n&quot;;
  
      print &quot;\nThis script removes duplicate records from a \&quot;short\&quot; format BLAST output file, and keeps only the \&quot;best\&quot; records.\nIt sorts by smallest e-value and then biggest percent identity.\n&quot;;
      print &quot;Usage: bestbenter code herelast.pl &lt;input file&gt; &lt;output file&gt;\n&quot;;
      print &quot;\n Author \n&quot;;
      print &quot;01/07/2014\n&quot;;
  }
  </code></pre>