title: Is this an efficient / appropriate use of a bloom filter?
body: <p>I have an API from which data comes in, but a lot of it is redundant (can be determined from an id). I have a bloom filter built up with a few million entries to start. </p>
  
  <p>I am using <a href="https://github.com/jaybaird/python-bloomfilter" rel="nofollow">this library</a> to handle the implementation.
  From my reading online:</p>
  
  <blockquote>
    <p>Bloom Filter is a compact data structure for probabilistic representation of a set of variables to ensure whether elements in set are certainly present or <strong>definitely not</strong> present in the set.  <a href="http://qr.ae/RglZcg" rel="nofollow">Source</a></p>
  </blockquote>
  
  <p>If I have some pseudocode like this</p>
  
  <pre><code>newData #some dataset
  for row in newData:
      #filter.add() returns True if in set, we want only not in set
      if not filter.add(row.id):
          #do some stuff with fresh data    
  </code></pre>
  
  <p>This function will be called every time a set of data comes in, which is about 200 new entries/sec. </p>
  
  <p>Is this an efficient way to use a bloom filter?</p>