<p>I am following the book Kubernetes for developers and seems maybe book is heavily outdated now.
Recently I have been trying to get prometheus up and running on kubernetes following the instruction from book. That suggested to install and use HELM to get Prometheus and grafana up and running.</p>
<pre><code> helm install monitor stable/prometheus --namespace monitoring
</code></pre>
<p>this resulted:</p>
<pre><code>NAME                                               READY   STATUS             RESTARTS   AGE   IP               NODE              NOMINATED NODE   READINESS GATES
monitor-kube-state-metrics-578cdbb5b7-pdjzw        0/1     CrashLoopBackOff   14         36m   192.168.23.1     kube-worker-vm3   &lt;none&gt;           &lt;none&gt;
monitor-prometheus-alertmanager-7b4c476678-gr4s6   0/2     Pending            0          35m   &lt;none&gt;           &lt;none&gt;            &lt;none&gt;           &lt;none&gt;
monitor-prometheus-node-exporter-5kz8x             1/1     Running            0          14h   192.168.1.13     rockpro64         &lt;none&gt;           &lt;none&gt;
monitor-prometheus-node-exporter-jjrjh             1/1     Running            1          14h   192.168.1.35     osboxes           &lt;none&gt;           &lt;none&gt;
monitor-prometheus-node-exporter-k62fn             1/1     Running            1          14h   192.168.1.37     kube-worker-vm3   &lt;none&gt;           &lt;none&gt;
monitor-prometheus-node-exporter-wcg2k             1/1     Running            1          14h   192.168.1.36     kube-worker-vm2   &lt;none&gt;           &lt;none&gt;
monitor-prometheus-pushgateway-6898f8475b-sk4dz    1/1     Running            0          36m   192.168.90.200   osboxes           &lt;none&gt;           &lt;none&gt;
monitor-prometheus-server-74d7dc5d4c-vlqmm         0/2     Pending            0          14h   &lt;none&gt;           &lt;none&gt;            &lt;none
</code></pre>
<p>For the prometheus server I checked why is it Pending:</p>
<pre><code># kubectl describe pod monitor-prometheus-server-74d7dc5d4c-vlqmm -n monitoring
Name:           monitor-prometheus-server-74d7dc5d4c-vlqmm
Namespace:      monitoring
Priority:       0
Node:           &lt;none&gt;
Labels:         app=prometheus
                chart=prometheus-13.8.0
                component=server
                heritage=Helm
                pod-template-hash=74d7dc5d4c
                release=monitor
Annotations:    &lt;none&gt;
Status:         Pending
IP:             
IPs:            &lt;none&gt;
Controlled By:  ReplicaSet/monitor-prometheus-server-74d7dc5d4c
Containers:
  prometheus-server-configmap-reload:
    Image:      jimmidyson/configmap-reload:v0.4.0
    Port:       &lt;none&gt;
    Host Port:  &lt;none&gt;
    Args:
      --volume-dir=/etc/config
      --webhook-url=http://127.0.0.1:9090/-/reload
    Environment:  &lt;none&gt;
    Mounts:
      /etc/config from config-volume (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from monitor-prometheus-server-token-n49ls (ro)
  prometheus-server:
    Image:      prom/prometheus:v2.20.1
    Port:       9090/TCP
    Host Port:  0/TCP
    Args:
      --storage.tsdb.retention.time=15d
      --config.file=/etc/config/prometheus.yml
      --storage.tsdb.path=/data
      --web.console.libraries=/etc/prometheus/console_libraries
      --web.console.templates=/etc/prometheus/consoles
      --web.enable-lifecycle
    Liveness:     http-get http://:9090/-/healthy delay=30s timeout=30s period=15s #success=1 #failure=3
    Readiness:    http-get http://:9090/-/ready delay=30s timeout=30s period=5s #success=1 #failure=3
    Environment:  &lt;none&gt;
    Mounts:
      /data from storage-volume (rw)
      /etc/config from config-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from monitor-prometheus-server-token-n49ls (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  config-volume:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      monitor-prometheus-server
    Optional:  false
  storage-volume:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  monitor-prometheus-server
    ReadOnly:   false
  monitor-prometheus-server-token-n49ls:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  monitor-prometheus-server-token-n49ls
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  &lt;none&gt;
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age                  From               Message
  ----     ------            ----                 ----               -------
  Warning  FailedScheduling  28m (x734 over 14h)  default-scheduler  0/6 nodes are available: 6 pod has unbound immediate PersistentVolumeClaims.
  Warning  FailedScheduling  3m5s (x23 over 24m)  default-scheduler  0/5 nodes are available: 5 pod has unbound immediate PersistentVolumeClaims.
r
</code></pre>
<p>However this message I am seeing <code>0/5 nodes are available: 5 pod has unbound immediate PersistentVolumeClaims.</code> is coming with all other nodejs's StatefulSets and rabbitmq Deployments I have  tried created.  for rabbitmq and nodejs I figured out I need to create a PersistantVolume and a storage class whose name I needed to specify in the PV and PVC. and then it all worked but now I have Prometheus Server, Do I have to do the same for prometheus as well ? why is it not instructed by the HELM ?</p>
<p>Has something change in the Kubernetes API recently ? that I always have to create a PV and Storage Class explicitly for a PVC ?</p>

<p>Unless you configure your cluster with <a href="https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/" rel="nofollow noreferrer">dynamic volume provisioning </a>, you will have to make the PV manually each time. Even if you are not on a cloud, you can setup dynamic storage providers. There are a number of options for providers and you can find many <a href="https://landscape.cncf.io/card-mode?category=cloud-native-storage&amp;grouping=category" rel="nofollow noreferrer">here</a>. Ceph and minio are popular providers.</p>
