<p>I'm running a Kubernetes job, where I want to monitor the state. I'm running various <code>--watch-only</code> commands simultaneously, e.g
<code>kubectl get pods --watch-only</code>, which shows me the updated state of pods. But, I want to have <em>timestamp</em> and some <em>string</em> appended to the output.</p>
<p>The idea is to know when the state changed and also add additional info as a string.</p>
<p>How can I achieve this?</p>

kaur
<p>Giving more visibility to the potential solution for the question posted in the comments by original poster:</p>
<blockquote>
<p>This is what i found working so far kubectl get pods --watch-only | while read line ; do echo -e &quot;$(date +&quot;%Y-%m-%d %H:%M:%S.%3N&quot;)\t pods\t $line&quot; ; done</p>
</blockquote>
<p>Command used:</p>
<ul>
<li><code>$ kubectl get pods --watch-only | while read line ; do echo -e &quot;$(date +&quot;%Y-%m-%d %H:%M:%S.%3N&quot;)\t pods\t $line&quot; ; done</code></li>
</ul>
<p>Solution is correct but the changes in state (<code>PENDING</code>,<code>RUNNING</code>,<code>SUCCEEDED/COMPLETED</code>) would need to be further extracted (assuming further actions) from the above command.</p>
<hr />
<p>Looking on this from a different perspective you can use the <a href="https://kubernetes.io/docs/reference/using-api/client-libraries/" rel="noreferrer">official Kubernetes API library</a> to monitor the statuses of pods and jobs and act accordingly to them (for example: do something when job succeeded).</p>
<hr />
<p>I've created an <strong>example</strong> app with Kubernetes Python API library to watch the statuses of pods and jobs.</p>
<p>Assuming that:</p>
<ul>
<li>You have a working Kubernetes cluster with <code>kubectl</code> configured</li>
<li>You have installed Python and required library:
<ul>
<li><code>$ pip install kubernetes</code></li>
</ul>
</li>
</ul>
<p>Using the example for a <code>Job</code>:</p>
<blockquote>
<p><em><a href="https://kubernetes.io/docs/concepts/workloads/controllers/job/" rel="noreferrer">Kubernetes.io: Docs: Concepts: Job</a></em></p>
</blockquote>
<h3>Example for pods</h3>
<p>Below is the sample code in Python3 that would watch for pods and print a message when status of pod is set to <code>Succeeded</code>:</p>
<pre class="lang-py prettyprint-override"><code>from kubernetes import client, config, watch
from datetime import datetime

config.load_kube_config()

v1 = client.CoreV1Api()
w = watch.Watch()
for event in w.stream(v1.list_namespaced_pod, namespace=&quot;default&quot;, watch=False):
    print(&quot;POD_NAME: &quot; + event['object'].metadata.name) # print the name
    print(&quot;TIME: &quot; + str(datetime.now())) # print the time 
    print(&quot;PHASE: &quot; +  event['object'].status.phase) # print the status of the pod
    print(&quot;CUSTOM AMAZING TEXT HERE!&quot;)

    if (event['object'].status.phase == &quot;Succeeded&quot;) and (event['type'] != &quot;DELETED&quot;): # do below when condition is met 
        print (&quot;----&gt; This pod succeeded, do something here!&quot;)
    print(&quot;---&quot;)
</code></pre>
<p>This would produce an output similar to one below:</p>
<pre class="lang-sh prettyprint-override"><code>POD_NAME: pi-pjmm5
TIME: 2020-09-06 15:28:01.541244
PHASE: Pending
CUSTOM AMAZING TEXT HERE!
---
POD_NAME: pi-pjmm5
TIME: 2020-09-06 15:28:03.894063
PHASE: Running
CUSTOM AMAZING TEXT HERE!
---
POD_NAME: pi-pjmm5
TIME: 2020-09-06 15:28:09.044219
PHASE: Succeeded
CUSTOM AMAZING TEXT HERE!
----&gt; This pod succeeded, do something here!
---
</code></pre>
<h3>Example for jobs</h3>
<p>Below is the sample code in Python3 that would watch for jobs and print a message when status of job is set to <code>Succeeded</code>:</p>
<pre class="lang-py prettyprint-override"><code>from kubernetes import client, config, watch
from datetime import datetime

config.load_kube_config()

v1 = client.BatchV1Api()
w = watch.Watch()
for event in w.stream(v1.list_namespaced_job, namespace=&quot;default&quot;, watch=False):
    print(&quot;JOB_NAME: &quot; + event['object'].metadata.name)
    print(&quot;TIME: &quot; + str(datetime.now()))
    print(&quot;STATUS: &quot; + event['type'])
    print(&quot;CUSTOM AMAZING TEXT HERE!&quot;)

    if (event['object'].status.succeeded == 1) and (event['type'] != &quot;DELETED&quot;): 
        print (&quot;----&gt; This job succeeded, do something here!&quot;)
    print(&quot;---&quot;)
</code></pre>
<p>This would produce an output similar to one below:</p>
<pre class="lang-sh prettyprint-override"><code>JOB_NAME: pi
TIME: 2020-09-06 15:32:49.909096
STATUS: ADDED
CUSTOM AMAZING TEXT HERE!
---
JOB_NAME: pi
TIME: 2020-09-06 15:32:49.936856
STATUS: MODIFIED
CUSTOM AMAZING TEXT HERE!
---
JOB_NAME: pi
TIME: 2020-09-06 15:32:56.998511
STATUS: MODIFIED
CUSTOM AMAZING TEXT HERE!
----&gt; This job succeeded, do something here!
---
</code></pre>
