<p>I have a Kubernetes controller written using client-go <a href="https://pkg.go.dev/k8s.io/client-go/informers" rel="nofollow noreferrer">informers</a> package. It maintains a watch on all Pods in the cluster, there are about 15k of these objects, and their YAML representation takes around 600 MB to print (I assume their in-memory representation is not that different.)</p>
<p>As a result, this (otherwise really small) controller watching Pods ends up with a huge memory footprint (north of 1 GiB). Even methods that you'd think offer a way of filtering, such as the one named like <a href="https://pkg.go.dev/k8s.io/client-go/informers#NewFilteredSharedInformerFactory" rel="nofollow noreferrer"><code>NewFilteredSharedInformerFactory</code></a> doesn't really give you a way to specify a predicate function that chooses which objects are stored in the in-memory <code>cache</code>.</p>
<p>Instead, that method in client-go offers a <a href="https://pkg.go.dev/k8s.io/client-go@v0.24.2/informers/internalinterfaces#TweakListOptionsFunc" rel="nofollow noreferrer"><code>TweakListOptionsFunc</code></a>. It helps you control <code>ListOptions</code> but my predicate unfortunately cannot be satisfied with a <code>labelSelector</code> or <code>fieldSelector</code>. I need to drop the objects when they arrive to the controller through a predicate function.</p>
<blockquote>
<p>Note: the predicate I have is something like <em>&quot;Pods that have an ownerReference by a DaemonSet&quot;</em> (which is <a href="https://stackoverflow.com/questions/59442065/how-to-use-kubernetes-fieldselector-to-query-ownerreferences">not possible with <code>fieldSelectors</code></a> –also another question of mine) and there's no <code>labelSelector</code> that can work in my scenario.</p>
</blockquote>
<p>How would I go about configuring an informer on Pods that only have DaemonSet owner references to reduce the memory footprint of my controller?</p>

ahmet alp balkan
<p>It appears that today if you use <a href="https://pkg.go.dev/k8s.io/client-go/informers#SharedInformerFactory" rel="nofollow noreferrer">SharedInformers</a>, there's no way to filters which objects to keep in the shared cache and which ones to discard.</p>
<p>I have found an <a href="https://github.com/kubernetes/kube-state-metrics/blob/734389481ab1a47422f07e80ef93b4359cadbaed/pkg/sharding/listwatch.go" rel="nofollow noreferrer">interesting code snippet</a> in kube-state-metrics project that opts into the lower-layer of abstraction of initiating <code>Watch</code> calls directly (which would normally be considered as an anti-pattern) and using <a href="https://pkg.go.dev/k8s.io/apimachinery@v0.25.0/pkg/watch#Filter" rel="nofollow noreferrer"><code>watch.Filter</code></a>, it decides whether to return an object from a Watch() call (to a cache/reflector or not).</p>
<p>That said, many controller authors might choose to not go down this path as it requires you to specify your own cache/reflector/indexer around the Watch() call. Furthermore, projects like <code>controller-runtime</code> don't even let you get access to this low-level machinery, as far as I know.</p>
<hr />
<p>Another aspect of reducing controllers' memory footprint can be done through field/data erasure on structs (instead of discarding objects altogether). This is possible in newer versions of <code>client-go</code> through <a href="https://pkg.go.dev/k8s.io/client-go@v0.24.2/tools/cache#TransformFunc" rel="nofollow noreferrer"><code>cache.TransformFunc</code></a>, which can let you delete fields of irrelevant objects (though, these objects would still consume some memory). This one is more of a band-aid that can make your situation better.</p>
<hr />
<p>In my case, I mostly needed to watch for DaemonSet Pods in certain namespaces, so I refactored the code from using 1 informer (watching all namespaces) to N namespace-scoped informers running concurrently.</p>
