<p>Recently I ran into a strange problem. We have two pods running into an openshift cluster that shares a persistent volume (<code>GlusterFs</code>) between them.
Now for the sake of this explanation, let's assume one of the pods was <em><strong>PodA</strong></em> and the Other was <em><strong>PodB</strong></em>, in this case, <em><strong>PodB</strong></em> was running for three months, there is automation in <em><strong>POdA</strong></em> which creates/updates files in the shared persistence volume and <em><strong>PodB</strong></em> reads it and perform some operation based on the input.</p>
<p>Now coming to the problem, whenever <em><strong>POdA</strong></em> created a new file in the shared PV it was visible and accessible from <em><strong>PodA</strong></em>. However, there were a few files that <em><strong>PodA</strong></em> was updating periodically, but the change was not reflected in <em><strong>PodB</strong></em>. So in <em><strong>PodB</strong></em>, we could only see the old version of those files. To solve that problem, we have forcefully deleted   <em><strong>PodB</strong></em>, and then openshift recreated it, and the problem was gone.</p>
<p>I thought in <code>PV</code> mechanism Kubernetes mount external storage/folder into the <code>pod</code> (container), and there is no intermediate layer or cache or something like that. From what we have experienced so far, it seems every container (or <code>pod</code>) creates a local copy of those files, or maybe there is a  cache in between (<code>PV</code> and <code>pod</code>),</p>
<p>I have searched about this on google and could not find a detailed explanation on how this <code>PV</code> mount works in Kubernetes , would love to know the actual reason behind this problem.</p>

Arko
<p>There is no caching mechanism for PVs provided by Kubernetes, so the problem you are observing must be located in either the GlusterFS CSI driver or GlusterFS itself.</p>
