<p>I'm trying to customize the behavior of the <code>kube-scheduler</code> on an AKS cluster (kubernetes v1.19.3), as described in <a href="https://kubernetes.io/docs/reference/scheduling/config/" rel="nofollow noreferrer">Scheduler Configuration</a>.</p>
<p>My goal is to use the <code>NodeResourcesMostAllocated</code> plugin in order to schedule the pods using the least number of nodes possible.</p>
<p>Consider the following file - <code>most-allocated-scheduler.yaml</code></p>
<pre><code>apiVersion: kubescheduler.config.k8s.io/v1beta1
kind: KubeSchedulerConfiguration
profiles:
  - schedulerName: default-scheduler
  - schedulerName: most-allocated-scheduler
    plugins:
      score:
        disabled:
        - name: NodeResourcesLeastAllocated
        enabled:
        - name: NodeResourcesMostAllocated
          weight: 2
</code></pre>
<p>According to the documentation, I can specify scheduling profiles by running something like:</p>
<pre><code>kube-scheduler --config most-allocated-scheduler.yaml
</code></pre>
<p>But where exactly can I find the <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/" rel="nofollow noreferrer">kube-scheduler</a> in order to run the above command? I'd like to do this ideally on a pipeline. Is it possible to do such thing when using AKS?</p>

Rui Jarimba
<p><a href="https://kubernetes.io/docs/concepts/scheduling-eviction/kube-scheduler/#kube-scheduler" rel="noreferrer">kube-scheduler</a> is a part of <a href="https://kubernetes.io/docs/concepts/overview/components/#control-plane-components" rel="noreferrer">kubernetes control plane</a>. It's components are scheduled on <strong>master node</strong>, to which on managed kubernetes solutions such as <strong>AKS</strong>, <strong>GKE</strong> or <strong>EKS</strong>, you have no access.</p>
<p>This means it's not possible to reconfigure your <code>kube-scheduler</code> on a running <strong>AKS</strong> cluster. Compare with <a href="https://github.com/Azure/AKS/issues/609#issuecomment-414100148" rel="noreferrer">this</a> answer on <strong>AKS's GitHub</strong> page.</p>
<p>However, it is possible to provide custom configuration for your kube-scheduler when creating a new cluster, using <a href="https://github.com/Azure/aks-engine/blob/master/docs/topics/clusterdefinitions.md" rel="noreferrer">cluster definitions</a>, specifically in <a href="https://github.com/Azure/aks-engine/blob/master/docs/topics/clusterdefinitions.md#schedulerconfig" rel="noreferrer">schedulerConfig</a> section:</p>
<blockquote>
<h3>schedulerConfig</h3>
<p><code>schedulerConfig</code>  declares runtime configuration for the
kube-scheduler daemon running on all master nodes. Like
<code>kubeletConfig</code>,  <code>controllerManagerConfig</code>, and  <code>apiServerConfig</code>
it is a generic key/value object, and a child property of
<code>kubernetesConfig</code>. An example custom apiserver config:</p>
<pre><code>&quot;kubernetesConfig&quot;: {
    &quot;schedulerConfig&quot;: {
        &quot;--v&quot;: &quot;2&quot;
    }
}
</code></pre>
<p>See
<a href="https://kubernetes.io/docs/reference/generated/kube-scheduler/" rel="noreferrer">here</a>
for a reference of supported kube-scheduler options.</p>
<p>...</p>
</blockquote>
<p>Keep in mind however that not all options are supported. Docs says that e.g. <code>--kubeconfig</code> is not supported, but as you can read <a href="https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/" rel="noreferrer">here</a>, this flag is deprecated anyway. There is nothing about <code>--config</code> flag so you can simply try if it works.</p>
<p>You can also achieve it by using <a href="https://github.com/Azure/aks-engine/blob/master/docs/topics/clusterdefinitions.md#custom-yaml-for-kubernetes-component-manifests" rel="noreferrer">Custom YAML for Kubernetes component manifests</a>:</p>
<blockquote>
<p>Custom YAML specifications can be configured for kube-scheduler,
kube-controller-manager, cloud-controller-manager and kube-apiserver
in addition to the addons described
<a href="https://github.com/Azure/aks-engine/blob/master/docs/topics/clusterdefinitions.md#addons" rel="noreferrer">above</a>.
You will need to pass in a  <em>base64-encoded</em>  string of the kubernetes
manifest YAML file to  <em>KubernetesComponentConfig[&quot;data&quot;]</em>  . For
example, to pass a custom kube-scheduler config, do the following:</p>
<pre><code>&quot;kubernetesConfig&quot;: {
    &quot;schedulerConfig&quot;: {
            &quot;data&quot; : &quot;&lt;base64-encoded string of your k8s manifest YAML&gt;&quot;
        }
}
</code></pre>
<blockquote>
<p><em><strong>NOTE</strong></em>: Custom YAML for addons is an experimental feature. Since  <code>Addons.Data</code>  allows you to provide your own scripts, you are
responsible for any undesirable consequences of their errors or
failures. Use at your own risk.</p>
</blockquote>
</blockquote>
<p>So as you can see, even in managed kubernetes solutions such as <strong>AKS</strong>, <code>kube-scheduler</code> can be customized to certain extent, but only when you create a new cluster.</p>
