<p>I've been trying to deploy a workflow in Argo with Kubernetes and I'm getting this error</p>
<p><a href="https://i.stack.imgur.com/BGPAk.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/BGPAk.png" alt="![Kubernetes Argo Error" /></a></p>
<p>Can someone help me to know the root of the issue?</p>
<p>I’ve tried several things but I’ve been unsuccessful.</p>

<p>The way Argo solves that problem is by using compression on the stored entity, but the real question is whether you have to have all 3MB worth of that data at once, or if it is merely more convenient for you and they could be decomposed into separate objects with relationships between each other. The kubernetes API is not a blob storage, and shouldn't be treated as one.</p>
<ul>
<li>The &quot;error&quot;: &quot;Request entity too large: limit is 3145728&quot; is probably
the default response from kubernetes handler for objects larger than
3MB, as you can see <a href="https://github.com/kubernetes/kubernetes/blob/db1990f48b92d603f469c1c89e2ad36da1b74846/test/integration/master/synthetic_master_test.go#L315" rel="nofollow noreferrer">here at L305</a> of the source code:</li>
</ul>
<p>expectedMsgFor1MB := <code>etcdserver: request is too large</code>
expectedMsgFor2MB := <code>rpc error: code = ResourceExhausted desc = trying to send message larger than max</code>
expectedMsgFor3MB := <code>Request entity too large: limit is 3145728</code>
expectedMsgForLargeAnnotation := <code>metadata.annotations: Too long: must have at most 262144 bytes</code></p>
<ul>
<li>The <a href="https://github.com/etcd-io/etcd/issues/9925" rel="nofollow noreferrer">ETCD</a> has indeed a 1.5MB limit for processing a file and you will
find on ETCD Documentation a suggestion to try the--max-request-bytes
flag but it would have no effect on a GKE cluster because you don't
have such permission on master node.</li>
</ul>
<p>But even if you did, it would not be ideal because usually this error means that you are <a href="https://github.com/kubeflow/pipelines/issues/3134#issuecomment-591278230" rel="nofollow noreferrer">consuming the objects</a> instead of referencing them which would degrade your performance.</p>
<p>I highly recommend that you consider instead these options:</p>
<p><strong>- Determine whether your object includes references that aren't used</strong></p>
<p><strong>- Break up your resource</strong></p>
<p><strong>- Consider a volume mount instead</strong></p>
<p>There's a request for <a href="https://github.com/kubernetes/kubernetes/issues/88709" rel="nofollow noreferrer">a new API Resource</a>: File (orBinaryData) that could apply to your case. It's very fresh but it's good to keep an eye on.</p>
<p>Partial source for this answer: <a href="https://github.com/etcd-io/etcd/issues/9925" rel="nofollow noreferrer">https://stackoverflow.com/a/60492986/12153576</a></p>
