<h3>Context</h3>
<p>I have a pod with two containers:</p>
<ul>
<li><code>main</code> whose simple job is to display the content of a directory</li>
<li><code>sidecar</code> whose responsibility is to synchronize the content of a blob storage into a predefined directory</li>
</ul>
<p>In order for the synchronization to be atomic, <code>sidecar</code> download the blob storage content into a new temp directory and then switch a symlink in the target directory.</p>
<p>The target directory is shared between the two containers using an <code>emptyDir</code> volume.</p>
<h3>Problem</h3>
<p><code>main</code> has the symlink but cannot list the content sitting behind.</p>
<h3>Question</h3>
<p>How to access the latest synchronized data?</p>
<h3>Additional information</h3>
<h4>Reason</h4>
<p>I try to achieve what is being done by <a href="https://airflow.apache.org/docs/apache-airflow/stable/executor/kubernetes.html" rel="nofollow noreferrer">Apache Airflow</a> with <a href="https://github.com/kubernetes/git-sync" rel="nofollow noreferrer">Git-Sync</a> but, instead of using Git, I need to synchronize files from an Azure Blob storage. This is necessary because (1) my content is mostly dynamic and (2) the <code>azureFile</code> volume type has some serious <a href="https://github.com/Azure/AKS/issues/223" rel="nofollow noreferrer">performance issues</a>.</p>
<h4>Sync routine</h4>
<pre class="lang-sh prettyprint-override"><code>declare -r container='https://mystorageaccount.dfs.core.windows.net/mycontainer'
declare -r destination='/shared/container'

declare -r temp_dir=&quot;$(mktemp -d)&quot;
azcopy copy --recursive &quot;$container/*&quot; &quot;$temp_dir&quot;

declare -r temp_file=&quot;$(mktemp)&quot;
ln -sf &quot;$temp_dir&quot; &quot;$temp_file&quot;
mv -Tf &quot;$temp_file&quot; &quot;$destination&quot;
</code></pre>
<p>What we end up with:</p>
<pre class="lang-sh prettyprint-override"><code>$ ls /shared
container -&gt; /tmp/tmp.doGz2U0QNy
$ ls /shared/container
file1.txt file2.txt
</code></pre>
<h3>Solution</h3>
<p>My initial attempt had two mistakes:</p>
<ol>
<li>The symlink target was not present in the volume</li>
<li>The symlink target pointed to an absolute path in the sidecar container so, from the point of view of the main container, the folder did not exist</li>
</ol>
<p>Here is the routine revised:</p>
<pre class="lang-sh prettyprint-override"><code>declare -r container='https://mystorageaccount.dfs.core.windows.net/mycontainer'
declare -r destination='/shared/container'
declare -r cache_dir=&quot;$(dirname $destination)&quot;

declare -r temp_dir=&quot;$(mktemp -d -p $cache_dir)&quot;
azcopy copy --recursive &quot;$container/*&quot; &quot;$temp_dir&quot;

ln -sf &quot;$(basename $temp_dir)&quot; &quot;$cache_dir/symlink&quot;
mv -Tf &quot;$cache_dir/symlink&quot; &quot;$destination&quot;

</code></pre>

flappy
<p>A symlink is just a special kind of file that contains a filename; it doesn't actually contain the file content in any meaningful way, and it doesn't have to point to a file that exists.  <a href="https://linux.die.net/man/1/mktemp" rel="nofollow noreferrer"><strong>mktemp</strong>(1)</a> by default creates directories in <code>/tmp</code>, which probably isn't in the shared volume.</p>
<p>Imagine putting a physical file folder in a physical file cabinet, writing <code>the third drawer at the very front</code> on a Post-It note, and driving to another building, and handing the note to a colleague.  The Post-It note (the symlink) still exists, but in the other building's (container filesystem's) context, the location it names isn't especially meaningful.</p>
<p>The easiest way around this is to ask <code>mktemp</code> to create the file directly in the destination volume, and then create a relative-path symlink.</p>
<pre class="lang-sh prettyprint-override"><code># extract the volume location (you may already have this)
volume_dir=$(dirname &quot;$destination&quot;)

# force the download location to be inside the volume
# (mktemp --tmpdir option)
temp_dir=$(mktemp -d --tmpdir &quot;$volume_dir&quot;)

# actually do the download
azcopy copy --recursive &quot;$container/*&quot; &quot;$temp_dir&quot;

# set the symlink to a relative-path symlink, since the directory
# and the link are in the same place; avoids problems if the volume
# is mounted in different places in the two containers
ln -sf $(basename &quot;$temp_dir&quot;) &quot;$destination&quot;
</code></pre>
