title: Ensuring local caching with Ignite on YARN
body: <p>I have a stream processing application written in Flink &amp; I want to use its internal key-value store from the state backend to compute streaming aggregates. Because I am dealing with a lot of aggregates, I would like to avoid maintaining them on-heap inside the Flink application like the memory-backed and file-backed implementations currently offer. In stead, I would like to maintain a cache of the state in Apache Ignite, which in turn could use the write-through &amp; read-through features to provide a more reliable back-up in HBase.</p>
  
  <p>Ideally, I would have a single local Ignite cache on every physical node that handles the state for all long-running Flink operators on that node. E.g. each node has a single Ignite node in an 8 GB container available, whether it is running 1 or 10 Flink operators.</p>
  
  <p>The problem is that I want both Flink and Ignite to run on YARN. Through consistent partitioning, I can ensure that the data in general is sent to the correct cache, and in case of failures etc., it can be refilled from HBase. The problem I'm facing though is that Ignite seems to request containers from YARN randomly, meaning I have no guarantee that there is in fact a local cache available, even if I set the amount of Ignite nodes exactly the same as the amount of physical nodes.</p>
  
  <p>Any suggestions on how to achieve a one Ignite node per physical node set up? </p>