title: Better/Faster to Loop through set or list?
body: <p>If I have a python list that is has many duplicates, and I want to iterate through each item, but not through the duplicates, is it best to use a set (as in <code>set(mylist)</code>, or find another way to create a list without duplicates?  I was thinking of just looping through the list and checking for duplicates but I figured that's what <code>set()</code> does when it's initialized.</p>
  
  <p>So if <code>mylist = [3,1,5,2,4,4,1,4,2,5,1,3]</code> and I really just want to loop through <code>[1,2,3,4,5]</code> (order doesn't matter), should I use <code>set(mylist)</code> or something else?</p>
  
  <p>An alternative is possible in the last example, since the list contains every integer between its min and max value, I could loop through <code>range(min(mylist),max(mylist))</code> or through <code>set(mylist)</code>.  Should I generally try to avoid using set in this case?  Also, would finding the <code>min</code> and <code>max</code> be slower than just creating the <code>set</code>?</p>
  
  <hr>
  
  <p>In the case in the last example, the <code>set</code> is faster:</p>
  
  <pre><code>from numpy.random import random_integers
  ids = random_integers(1e3,size=1e6)
  
  def set_loop(mylist):
      idlist = []
      for id in set(mylist):
          idlist.append(id)
      return idlist
  
  def list_loop(mylist):
      idlist = []
      for id in range(min(mylist),max(mylist)):
          idlist.append(id)
      return idlist
  
  %timeit set_loop(ids)
  #1 loops, best of 3: 232 ms per loop
  
  %timeit list_loop(ids)
  #1 loops, best of 3: 408 ms per loop
  </code></pre>