<p>I am deploying some Flink jobs which require access to some services under a service mesh implemented via Linkerd and I'm running into this error:</p>
<pre><code>java.lang.NoClassDefFoundError: Could not initialize class foo.bar.Job 
</code></pre>
<p>I can confirm that the jar file contains the class that cannot be found apparently, so it's not a problem with the jar itself, but seems to be related to Linkerd. In particular, I'm using the following pod annotations for both the jobmanager and the taskmanager pods (taken from my Helm Chart values file):</p>
<pre><code>podAnnotations:
  linkerd.io/inject: enabled
  config.linkerd.io/skip-outbound-ports: 6123,6124
  config.linkerd.io/proxy-await: enabled
</code></pre>
<p>For what it's worth, I'm using the <a href="https://www.ververica.com/" rel="nofollow noreferrer">Ververica Platform</a> (Community Edition) for deploying my jobs to Kubernetes, although I don't think the issue is VVP-specific:</p>
<pre><code>{{- define &quot;vvp.deployment&quot; }}
kind: Deployment
apiVersion: v1
metadata:
  name: my-job
spec:
  template:
    spec:
      artifact:
        kind: jar
        flinkImageRegistry: {{ .Values.flink.imageRegistry }}
        flinkVersion: &quot;1.15.1&quot;
        flinkImageTag: 1.15.1-stream1-scala_2.12-java11-linkerd
        entryClass: foo.bar.Job
      kubernetes:
        jobManagerPodTemplate:
          metadata:
            {{- with .Values.flink.podAnnotations }}
            annotations:
              {{- toYaml . | nindent 14 }}
            {{- end }}
          spec:
            containers:
              - name: flink-jobmanager
                command:
                  - linkerd-entrypoint.sh
        taskManagerPodTemplate:
          metadata:
            {{- with .Values.flink.podAnnotations }}
            annotations:
              {{- toYaml . | nindent 14 }}
            {{- end }}
{{- end }}
</code></pre>
<p>where the contents of <code>linkerd-entrypoint.sh</code> are:</p>
<pre><code>#!/bin/bash
set -e
exec linkerd-await --shutdown -- &quot;$@&quot;
</code></pre>
<p>For extra context, the VVP and the flink jobs are deployed into different namespaces. Also, for the VVP pods, I'm not using any linkerd annotations whatsoever.</p>
<p>Has anyone encountered similar problems? The closest troubleshooting resource/guide that I've found so far is <a href="https://ververica.zendesk.com/hc/en-us/articles/7233687970460-How-to-integrate-Istio-with-Ververica-Platform" rel="nofollow noreferrer">this one</a>, which targets Istio instead of Linkerd.</p>

<p>Answering to myself after having determined the root cause of the issue.</p>
<p>Regarding Linkerd, everything was correctly setup. The main precaution that one needs to take is adding the <code>linkerd-await</code> binary to the Flink image and making sure to override the entrypoint for the jobmanager since otherwise you will run into issues when upgrading your jobs. The jobmanager won't kill the Linkerd proxy, and because of that it will hang around with <code>NotReady</code> status. Again, that is easily solved by wrapping the main cmd in a <code>linkerd-await</code> call. So, first add the <code>linkerd-await</code> binary to your docker image:</p>
<pre><code># Add linkerd-await and linkerd-entrypoint.sh
USER root
RUN apt-get update &amp;&amp; apt-get install -y wget
RUN wget https://github.com/linkerd/linkerd-await/releases/download/release%2Fv0.2.7/linkerd-await-v0.2.7-amd64 -O ./linkerd-await &amp;&amp; chmod +x ./linkerd-await
COPY scripts/flink/linkerd-entrypoint.sh ./linkerd-entrypoint.sh
</code></pre>
<p>Then, for the jobmanager only, override the entrypoint like this:</p>
<pre class="lang-yaml prettyprint-override"><code>spec:
  containers:
    - name: flink-jobmanager
      command:
       - linkerd-entrypoint.sh # defined above
</code></pre>
<p>Alternatively one could use the <code>LINKERD_DISABLED</code> or <code>LINKERD_AWAIT_DISABLED</code> env vars for bypassing the <code>linkerd-await</code> wrapper. For more info on using jobs &amp; Linkerd consult the following resources:</p>
<ul>
<li><a href="https://itnext.io/three-ways-to-use-linkerd-with-kubernetes-jobs-c12ccc6d4c7c" rel="nofollow noreferrer">https://itnext.io/three-ways-to-use-linkerd-with-kubernetes-jobs-c12ccc6d4c7c</a> (solution #3 is the one explained here)</li>
<li><a href="https://github.com/linkerd/linkerd-await" rel="nofollow noreferrer">https://github.com/linkerd/linkerd-await</a></li>
</ul>
<p>Also, regarding the annotation</p>
<pre class="lang-yaml prettyprint-override"><code>config.linkerd.io/proxy-await: enabled
</code></pre>
<p>, it does only the waiting but not the shutdown part, so if we are going to manually run <code>linkerd-await --shutdown -- &quot;$@&quot;</code> anyway, that annotation can be safely removed since it's redundant:</p>
<ul>
<li><a href="https://github.com/linkerd/linkerd2/issues/8006" rel="nofollow noreferrer">https://github.com/linkerd/linkerd2/issues/8006</a></li>
</ul>
<p>Finally, regarding:</p>
<pre><code>java.lang.NoClassDefFoundError: Could not initialize class foo.bar.Job 
</code></pre>
<p>let me clarify that this had nothing to do with Linkerd. This was mostly a config error along the lines of:</p>
<ul>
<li><a href="https://stackoverflow.com/questions/7325579/java-lang-noclassdeffounderror-could-not-initialize-class-xxx">java.lang.NoClassDefFoundError: Could not initialize class XXX</a></li>
</ul>
<p>Essentially (the specific details are irrelevant), there were some env vars missing in the taskmanager pods. Note that the exception message says &quot;Could not initialize class foo.bar.Job&quot; which is different from &quot;Could not find class...&quot;.</p>
<p>Sorry for the confusion!</p>
