title: Bloom filters in a distributed environment
body: <p>I have a system consisting of a few application instances, written in Java. Requests to them are load balanced for high availability. Every second, hundreds of small chunks of data (each consisting of a few simple strings) are received by this "cluster", stored in the database, kept for a couple of days and then discarded. Apart from storing this data, the main requirement is to quickly determine if a given value is stored in the database or not. An appropriately indexed and partitioned database table seems suited to the problem and does it's job well, at least for now. </p>
  
  <p>The problem is, about 80% of the searched values are not found because they are not in the database. Therefore I would like to speed things up a bit and make the search faster and less resource-intensive. A bloom filter would be the obvious choice, were it not for the fact that different application instances receive different parts of data and if each application instance has only a part of the data in it's bloom filter then those bloom filters are useless. </p>
  
  <p>Do you have any suggestions/ideas on how to solve this problem?</p>