<p>i'm trying to test the <a href="https://github.com/kubernetes-incubator/external-storage/tree/master/local-volume" rel="noreferrer">local persistent volume</a> in kubernetes v1.9.2.</p>

<p>from what i gather (and i may be wrong!) i cannot use <code>kubeadm</code> to add these feature gates:</p>

<pre><code>$ sudo kubeadm version
kubeadm version: &amp;version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.2", GitCommit:"5fa2db2bd46ac79e5e00a4e6ed24191080aa463b", GitTreeState:"clean", BuildDate:"2018-01-18T09:42:01Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}

$ kubeadm init --help
...
      --feature-gates string                    A set of key=value pairs that describe feature gates for various features. Options are:
        CoreDNS=true|false (ALPHA - default=false)
        DynamicKubeletConfig=true|false (ALPHA - default=false)
        SelfHosting=true|false (ALPHA - default=false)
        StoreCertsInSecrets=true|false (ALPHA - default=false)
...
</code></pre>

<p>sooo... i do a normal <code>kubeadm</code> init and then proceed to hack:</p>

<pre><code>/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
</code></pre>

<p>with </p>

<pre><code>Environment="KUBELET_FEATURE_GATES_ARGS=--feature-gates=PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true"
ExecStart=
ExecStart=/usr/bin/kubelet $KUBELET_KUBECONFIG_ARGS $KUBELET_SYSTEM_PODS_ARGS $KUBELET_NETWORK_ARGS $KUBELET_DNS_ARGS $KUBELET_AUTHZ_ARGS $KUBELET_CADVISOR_ARGS $KUBELET_CGROUP_ARGS $KUBELET_CERTIFICATE_ARGS $KUBELET_EXTRA_ARGS $KUBELET_FEATURE_GATES_ARGS
</code></pre>

<p>and reload/restart <code>kubelet</code>.</p>

<p>okay... let try creating the pv:</p>

<pre><code>$ cat local-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: example-local-pv
  annotations:
    "volume.alpha.kubernetes.io/node-affinity": '{
      "requiredDuringSchedulingIgnoredDuringExecution": {
        "nodeSelectorTerms": [
          { "matchExpressions": [
            { "key": "kubernetes.io/hostname",
              "operator": "In",
              "values": ["dhcp-nebula-129-230"]
            }
          ]}
         ]}
        }'
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /mnt/disks/fs2

$ kubectl create -f local-pv.yaml
The PersistentVolume "example-local-pv" is invalid:
* metadata.annotations: Forbidden: Storage node affinity is disabled by 
</code></pre>

<p>feature-gate
    * spec.local: Forbidden: Local volumes are disabled by feature-gate</p>

<p>aha! i say... i have to change the <code>kube-apiserver</code> too!</p>

<p>so i edit <code>/etc/kubernetes/manifests/kube-apiserver.yaml</code> and append the following to the <code>Command</code>:</p>

<pre><code>--feature-gates=PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true
</code></pre>

<p>and the api server dies and i'm stuck without <code>kubectl</code> as the <code>kubeapi-server</code> dies. :( </p>

<p>help?</p>

<p>You need to set the feature gates on api, scheduler and controller in a kubeadm config:</p>

<pre><code>apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
apiServerExtraArgs:
  service-node-port-range: 80-32767
  feature-gates: "PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true"
controllerManagerExtraArgs:
  feature-gates: "PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true"
schedulerExtraArgs:
  feature-gates: "PersistentLocalVolumes=true,VolumeScheduling=true,MountPropagation=true"
</code></pre>

<p>Storage class example:</p>

<pre><code>apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: mysql-data
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
</code></pre>

<p>PVC example:</p>

<pre><code>apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  labels:
    app: mariadb
  name: mysql-mariadb-0
  namespace: mysql
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: mysql-data
  selector:
    matchLabels:
      pod-name: mariadb-0
</code></pre>

<p>PV example:</p>

<pre><code>apiVersion: v1
kind: PersistentVolume
metadata:
  name: mysql-mariadb-0
  labels:
    pod-name: mariadb-0
  annotations:
    "volume.alpha.kubernetes.io/node-affinity": '{
      "requiredDuringSchedulingIgnoredDuringExecution": {
        "nodeSelectorTerms": [
          { "matchExpressions": [
              { "key": "kubernetes.io/hostname",
                "operator": "In",
                "values": ["prod-mysql-0"]
              }
          ]}
        ]}}'
spec:
  capacity:
    storage: 2Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: mysql-data
  local:
    path: /mnt/local-storage/mysql-data-0
</code></pre>
