title: Hazelcast write-through map store pre-population on startup
body: <p>I am currently doing a <code>POC</code> for developing a <code>distributed, fault tolerant, ETL ecosystem</code>. I have selected <code>Hazelcast</code> for for my <code>clustering (data+notification)</code> purpose. Googling through Hazelcast resources took me <a href="http://highscalability.com/blog/2011/4/12/caching-and-processing-2tb-mozilla-crash-reports-in-memory-w.html" rel="nofollow">to this link</a> and it exactly matches how I was thinking to go about, using a map based solution. </p>
  
  <p>I need to understand one point. Before that, allow me to give a canonical idea of our architecture:</p>
  
  <p>Say <code>we have 2 nodes A,B running our server instance clustered through hazelcast</code>. One of them is a listener accepting requests (but can change on a fail over), say A.</p>
  
  <p>A gets a request and puts it to a distributed map. This map is write-through backed by a persistent store and a single memory backup is configured on nodes.</p>
  
  <p>Each instance has a local map entry listener, which on entry added event, would <code>(asynchronous/queuing)</code> process that entry and then remove it from the distributed map.</p>
  
  <p>This is working as expected.</p>
  
  <p><strong>Question:</strong></p>
  
  <p>Say 10 requests have been received and distributed with 5 on each nodes. 2 entries on each node has been processed and now both instance crashes.</p>
  
  <p>So there are total 6 entries present in the backing datastore now.</p>
  
  <p>Now we bring up both the instances. As per documentation - "<code>As of 1.9.3 MapLoader has the new MapLoader.loadAllKeys API. It is used for pre-populating the in-memory map when the map is first touched/used"</code></p>
  
  <p>We implement <code>loadAllKeys()</code> by simply loading all the key values present in the store.</p>
  
  <ol>
  <li><p>So does that mean there is a possibility where, both the instances will now load the 6 entries and process them (thus resulting in duplicate processing)? <strong>Or</strong> is it handled in a synchronized way so that loading is done only once in a cluster?</p></li>
  <li><p><code>On server startup I need to process the pending entries</code> (if any). I see that the data is loaded, however the entryAdded event is not fired. How can make the entryAdded event fire (or any other elegant way, by which I will know that there are pending entries on startup)?</p></li>
  </ol>
  
  <p>Requesting suggestions.</p>
  
  <p>Thanks,
  Sutanu</p>