<p>Openshift provides by-default &quot;node-tuning-operator&quot; for tuning down system.</p>
<p>We can create our custom profiles using custom resource (CR).</p>
<p>But, the custom profiles are not being loaded/activated by the operator.</p>
<p>Instead of activating my custom profile, it is activating default profiles provided by openshift.</p>
<p>I am still working on to figure out correct profile configuration.</p>
<p>What may be the issue that tuned operator is not activating my custom profiles?</p>

Pankaj Yadav
<p>Documentation for tuned operator can be found at <a href="https://docs.openshift.com/container-platform/4.7/scalability_and_performance/using-node-tuning-operator.html" rel="nofollow noreferrer">https://docs.openshift.com/container-platform/4.7/scalability_and_performance/using-node-tuning-operator.html</a>.</p>
<p><strong>Generic Information about Tuned Operator:</strong></p>
<ul>
<li>Namespace/Project :   openshift-cluster-node-tuning-operator</li>
<li>Operator          :   cluster-node-tuning-operator</li>
<li>DaemonSet         :   tuned</li>
<li>CRD               :   tuneds.tuned.openshift.io</li>
<li>CR                :   Tuned/default &amp; Tuned/rendered</li>
</ul>
<p>The documentation says that we can create our own custom resources of <strong>kind=Tuned</strong> apart from default resources provided by openshift named &quot;<strong>Tuned/default &amp; Tuned/rendered</strong>&quot;.</p>
<p>These resources provide default profiles named <strong>&quot;openshift&quot;, &quot;openshift-node&quot; and &quot;openshift-control-plane&quot;</strong>.</p>
<p>More information can be seen using below command:</p>
<pre><code>oc get Tuned/default -n openshift-cluster-node-tuning-operator -o yaml
</code></pre>
<p>Now, we can create our own custom profile as part of custom resource to tune our own settings.</p>
<p>The trick here is that the configuration in custom resource yaml file regarding custom profile should be correct. If it is correct, tuned operator will load the profile and activate it. If it is incorrect, then tuned operator will NOT activate it and it will ignore any future correct configuration also.</p>
<p>This is a bug in tuned operator which is addressed as part of <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1919970" rel="nofollow noreferrer">https://bugzilla.redhat.com/show_bug.cgi?id=1919970</a>.</p>
<p><strong>Fix:</strong> Upgrade openshift cluster version to 4.7 and above.</p>
<p><strong>Workaround:</strong> Delete the tuned pod so that operator will create new pod. Once new pod is created, it will activate correct profile. (Hoping configuration in your CR.yaml was corrected).</p>
<p><strong>Important Commands:</strong></p>
<ul>
<li>To find out pod on which tuned operator itself is running:</li>
</ul>
<blockquote>
<p>oc get pod -n openshift-cluster-node-tuning-operator -o wide</p>
</blockquote>
<ul>
<li>To check logs of the operator pod: (Actual pod name can be found from
above command)</li>
</ul>
<blockquote>
<p>oc logs pod/cluster-node-tuning-operator-6644cd48bb-z2qxn -n
openshift-cluster-node-tuning-operator</p>
</blockquote>
<ul>
<li>To check which all Custom Resouces of kind=Tuned are present:</li>
</ul>
<blockquote>
<p>oc get Tuned -n openshift-cluster-node-tuning-operator</p>
</blockquote>
<ul>
<li>To describe and check default profiles:</li>
</ul>
<blockquote>
<p>oc get Tuned/default -n openshift-cluster-node-tuning-operator -o yaml</p>
</blockquote>
<ul>
<li>To find out all tuned pods and nodes on which they are running in the
cluster:</li>
</ul>
<blockquote>
<p>oc get pod -n openshift-cluster-node-tuning-operator -o wide</p>
</blockquote>
<ul>
<li>To check logs of a particular tuned pod: (Actual pod name can be
found from above command)</li>
</ul>
<blockquote>
<p>oc logs tuned-h8xgh -n openshift-cluster-node-tuning-operator -f</p>
</blockquote>
<ul>
<li>To login into tuned pod and manually confirm tuning is applied or
not: (Actual pod name can be found from previous-to-above command)</li>
</ul>
<blockquote>
<p>oc exec -it tuned-h8xgh -n openshift-cluster-node-tuning-operator  --bash</p>
</blockquote>
<ul>
<li><p>You can execute below commands after login into tuned pod using above
command to verify tuning settings:</p>
<blockquote>
<p>bash-4.4# cat /etc/tuned/infra-nodes/tuned.conf<br />
[main]
summary=Optimize systems running OpenShift Infra nodes<br />
[sysctl]
fs.inotify.max_user_watches = 1048576
vm.swappiness = 1</p>
</blockquote>
<blockquote>
<p>bash-4.4# tuned-adm recommend    Cannot talk to Tuned daemon via DBus.
Is Tuned daemon running?    infra-nodes    bash-4.4#</p>
</blockquote>
</li>
</ul>
<blockquote>
<pre><code>bash-4.4# tuned-adm active 
Cannot talk to Tuned daemon via DBus. Is Tuned daemon running? 
Current active profile: openshift-control-plane 
bash-4.4#
</code></pre>
</blockquote>
<p><strong>Note:</strong> The above sample code exactly depicts the issue asked in this question. If you notice, active profile is &quot;openshift-control-plane&quot; whereas recommended/loaded one is &quot;infra-nodes&quot;. This is due to the existing bug as mentioned previously. Once you delete tuned pod (tuned-h8xgh), operator will recover and activate correct profile.</p>
<p><strong>Sample issue in custom profile configuration:</strong>
If profile priorities are same as default profiles, then operator will give warning something similar as below:</p>
<pre><code>W0722 04:24:25.490704       1 profilecalculator.go:480] profiles openshift-control-plane/infra-node have the same priority 30, please use a different priority for your custom profiles!
</code></pre>
