<p>I am trying to transfer logs over to S3 just before the pod is terminated. For this, we need to</p>

<ol>
<li><p>Configure our container to have AWS-CLI. I did this successfully
using a script in postStart hook.</p></li>
<li><p>Execute AWS S3 command to transfer files from a hostPath to S3
bucket. Almost had this one !!!</p></li>
</ol>

<p>Here is my Kube Deployment (running on minikube):</p>

<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: logtransfer-poc
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: logs
    spec:
      volumes:
      - name: secret-resources
        secret:
          secretName: local-secrets
      - name: testdata
        hostPath:
          path: /data/testdata
      containers:
        - name: logtransfer-poc
          image: someImage
          ports:
          - name: https-port
            containerPort: 8443
          command: ["/bin/bash","-c","--"]
          args: ["while true; do sleep 30; done;"]
          volumeMounts:
          - name: secret-resources
            mountPath: "/data/apache-tomcat/tomcat/resources"
          - name: testdata
            mountPath: "/data/testdata"
          lifecycle:
            postStart:
              exec:
                command: ["/bin/sh", "-c", "cd /data/testdata/ &amp;&amp; chmod u+x installS3Script.sh &amp;&amp; ./installS3Script.sh &gt; postInstall.logs"]
            preStop:
              exec:
                command: ["/bin/sh", "-c", "cd /data/testdata/ &amp;&amp; chmod u+x transferFilesToS3.sh &amp;&amp; ./transferFilesToS3.sh &gt; postTransfer.logs"]
          terminationMessagePath: /data/testdata/termination-log
      terminationGracePeriodSeconds: 30
      imagePullSecrets:
        - name: my-docker-credentials</code></pre>
</div>
</div>
</p>

<p>installS3Script.sh</p>

<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>#!/bin/bash

apt-get update
curl -O https://bootstrap.pypa.io/get-pip.py
python3 get-pip.py --user
chmod u+x get-pip.py
echo "PATH=$PATH:/root/.local/bin" &gt;&gt; ~/.bashrc &amp;&amp; echo "Path Exported !!"
source ~/.bashrc &amp;&amp; echo "Refreshed profile !"
pip3 install awscli --upgrade --user
mkdir -p ~/.aws
cp /data/testdata/config/config ~/.aws
cp /data/testdata/config/credentials ~/.aws</code></pre>
</div>
</div>
</p>



<p>transferFilesToS3.sh</p>

<p><div class="snippet" data-lang="js" data-hide="false" data-console="true" data-babel="false">
<div class="snippet-code">
<pre class="snippet-code-html lang-html prettyprint-override"><code>#!/bin/bash

# export AWS_DEFAULT_PROFILE=admin
echo "Transfering files to S3.."
aws s3 cp /data/testdata/data s3://testpratham --recursive --profile admin
aws s3 ls s3://testpratham --profile admin
echo "Transfer to S3 successfull !!"</code></pre>
</div>
</div>
</p>

<p>What failed: the transferFilesToS3.sh runs successfully BUT its does NOT execute the AWS commands..</p>

<p>What works: I created test-log files and put the aws commands in postStart hook (installS3Script.sh) and it works fine !!</p>

<p>I think I might be looking into preStop hooks differently. I read a few articles on <a href="https://blog.openshift.com/kubernetes-pods-life/" rel="nofollow noreferrer">lifecycle</a> and <a href="https://pracucci.com/graceful-shutdown-of-kubernetes-pods.html" rel="nofollow noreferrer">preStop</a> hook. Also had a relative question on use of <a href="https://stackoverflow.com/questions/54676124/relation-between-prestop-hook-and-terminationgraceperiodseconds/54680573#54680573">preStop hook with grace period</a>.</p>

<p>Any suggestions/help on what I might be missing are appreciated.</p>

<p>Maybe it would be easier using <a href="https://github.com/nuvo/skbn" rel="nofollow noreferrer">Skbn</a>.</p>

<blockquote>
  <p><strong>Skbn</strong> is a tool for copying files and directories between Kubernetes and cloud storage providers. It is named after the 1981 video game  <a href="https://en.wikipedia.org/wiki/Sokoban" rel="nofollow noreferrer">Sokoban</a>. Skbn uses an in-memory buffer for the copy process, to avoid excessive memory consumption. Skbn currently supports the following providers:
  -   AWS S3
  -   Minio S3
  -   Azure Blob Storage</p>
</blockquote>

<p>You could use:</p>

<pre><code>skbn cp \
    --src k8s://&lt;namespace&gt;/&lt;podName&gt;/&lt;containerName&gt;/&lt;path&gt; \
    --dst s3://&lt;bucket&gt;/&lt;path&gt;
</code></pre>

<p>You should look at <a href="https://github.com/nuvo/skbn/tree/master/examples/in-cluster" rel="nofollow noreferrer">in-cluster</a> usage, as it will require setting up ClusterRole, ClusterRoleBinding and ServiceAccount.</p>
