title: How to better utilize local cache with load balancing strategies?
body: <p>I have an Authentication service where I need to cache some user information for better performance. I chose to use local cache because Authentication service probably will be called on each request so I want it to be super fast. Compared to remote cache options local cache is a lot faster (local cache access is below 1ms while remote cache access is around 25ms).</p>
  <p>The problem is I can not cache as much information as a distributed cache without running out of memory (talking about millions of users). I can either leave it as it is and when local cache reaches the memory limit it would evict some other data but that would be bad optimization of the cache. Or I can use some kind of load balancer strategy where users will be redirected to same Authentication service instances based on their IP address or other criteria thus the cache hits will be a lot higher.</p>
  <p>It kind of defeats the purpose of having stateless services however I think I can slightly compromise from this principle in network layer if I want both consistency and availability. And as for Authentication both are crucial for full security (user info always has to be up-to-date and available).</p>
  <p>What kind of load balancing techniques out there for solving this kind of problem? Can there be other solutions?</p>
  <p><strong>Note:</strong> Even though this question is specific to Authentication I think many other services that are frequently accesses and requires speed can benefit a lot from using local caches.</p>