<p>I have a flask api and I am trying to improve it identifying which function calls in the api definition takes the longest time whenever call it. For that I am using a profiler as highlighted in this <a href="https://dev.to/yellalena/profiling-flask-application-to-improve-performance-4970" rel="nofollow noreferrer">repo</a>. Whenever I make the api call, this profiler generates a .prof file which I can use with <code>snakeviz</code> to visualize.</p>
<p>Now I am trying to run this on aws cluster in the same region where my database is stored to minimize network latency time. I can get the api server running and make the api calls, my question is how can I transfer the <code>.prof</code> file from kubernetes pod without disturbing the api server. Is there a way to start a separate shell that transfers file to say an s3 bucket whenever that file is created without killing off the api server.</p>

monte
<p>If you want to automate this process or it's simply hard to figure out connectivity for running <code>kubectl exec ...</code>, one idea would be to use <a href="https://kubernetes.io/docs/concepts/workloads/pods/#how-pods-manage-multiple-containers" rel="nofollow noreferrer">a sidecar container</a>. So your pod contains two containers with a single <code>emptyDir</code> volume mounted into both. <code>emptyDir</code> is <a href="https://kubernetes.io/docs/concepts/storage/volumes/#emptydir" rel="nofollow noreferrer">perhaps the easiest way</a> to create a folder shared between all containers in a pod.</p>
<ul>
<li>First container is your regular Flask API</li>
<li>Second container is watching for new files in shared folder. Whenever it finds a file there it uploads this file to S3</li>
</ul>
<p>You will need to configure profiler so it dumps output into a shared folder.
One benefit of this approach is that you don't have to make any major modifications to the existing container running Flask.</p>
