title: Kubernetes pods getting different CPU limits
body: <p>We have several applications in a Kubernetes cluster that use Apache Ignite. Ignite creates various thread pools that are sized as follows:</p>
  
  <pre><code>Math.max(8, Runtime.getRuntime().availableProcessors())
  </code></pre>
  
  <p>So basically the thread pool will always have at least size 8, but could be more if the system believes there are more processors.</p>
  
  <p>The problem we're having is that some pods are spinning up with pool size 8, and others are using size 36, which is the number of CPUs on the node.</p>
  
  <p>We're using Helm to deploy all apps, but we're <strong>not</strong> setting any CPU limits for any pods. In theory they should all see the same number of available CPUs.</p>
  
  <p>What else could cause pods on the same node to see different views of how many processors are available?</p>
  
  <h2>Update</h2>
  
  <p>We have a health end point in all of our apps that shows the number of CPUS reported by the JVM, using the same <code>Runtime#availableProcessors()</code> method that Ignite uses.</p>
  
  <p>All of our apps, including ones where Ignite thinks there are 36 CPUs, report 2 processors once the process has started.</p>
  
  <p>I found this interesting line in the Java documentation for that method:</p>
  
  <blockquote>
    <p>This value may change during a particular invocation of the virtual machine. Applications that are sensitive to the number of available processors should therefore occasionally poll this property and adjust their resource usage appropriately.</p>
  </blockquote>
  
  <p>It seems we're in a race condition where early on in app startup, that value reports 36 but at some point drops to 2. Depending on when the Ignite beans are fired, they're seeing either 36 or 2.</p>