text: user9120: I'm attempting to remove redundant sequences from a fasta file (from NCBI). When I execute this code, it returns the number of spots, not the number of sequences. (Number of spots: 408,293, Number of sequences: 40,830)
  Please let me know if you have run a code that can remove the duplicate ideas or have any ideas/suggestions.
  Thank you SO much.
  from Bio import SeqIO
  import time
  
  start = time.time() 
  
  seen = []
  records = []
  
  for record in SeqIO.parse("b4r2.fasta", "fasta"):  
      if str(record.seq) not in seen:
          seen.append(str(record.seq))
          records.append(record)
  
  
  #writing to a fasta file
  SeqIO.write(records, "no_dupes_b4r2.fasta", "fasta")
  end = time.time()
  
  print(f"Run time is {(end- start)/60}") 
  
  
  user104: Your code (as currently formatted) should generate the correct output, but it is not efficient. Make seen a set to improve these O(n) in calls to O(1) time complexity- this will speed it up a lot
  seen = set()
  records = []
  
  for record in SeqIO.parse("b4r2.fasta", "fasta"):  
      if record.seq not in seen:
          seen.add(record.seq)
          records.append(record)
  
  
original_text: 9120: <p>I'm attempting to remove redundant sequences from a fasta file (from NCBI). When I execute this code, it returns the number of spots, not the number of sequences. (Number of spots: 408,293, Number of sequences: 40,830)</p>
  <p>Please let me know if you have run a code that can remove the duplicate ideas or have any ideas/suggestions.</p>
  <p>Thank you SO much.</p>
  <pre><code>from Bio import SeqIO
  import time
  
  start = time.time() 
  
  seen = []
  records = []
  
  for record in SeqIO.parse(&quot;b4r2.fasta&quot;, &quot;fasta&quot;):  
      if str(record.seq) not in seen:
          seen.append(str(record.seq))
          records.append(record)
  
  
  #writing to a fasta file
  SeqIO.write(records, &quot;no_dupes_b4r2.fasta&quot;, &quot;fasta&quot;)
  end = time.time()
  
  print(f&quot;Run time is {(end- start)/60}&quot;) 
  </code></pre>
   104: <p>Your code (as currently formatted) should generate the correct output, but it is not efficient. Make <code>seen</code> a set to improve these O(n) <code>in</code> calls to O(1) time complexity- this will speed it up a lot</p>
  <pre><code>seen = set()
  records = []
  
  for record in SeqIO.parse(&quot;b4r2.fasta&quot;, &quot;fasta&quot;):  
      if record.seq not in seen:
          seen.add(record.seq)
          records.append(record)
  </code></pre>
  