<p>I am trying to install PGO operator by following <a href="https://access.crunchydata.com/documentation/postgres-operator/5.3.0/quickstart/#installation" rel="nofollow noreferrer">this</a> Docs. When I run this command</p>
<pre><code>kubectl apply --server-side -k kustomize/install/default
</code></pre>
<p>my Pod run and soon it hit to crash loop back.</p>
<p><strong>What I have done</strong>
I check the logs of Pods with this command</p>
<pre><code>kubectl logs pgo-98c6b8888-fz8zj -n postgres-operator
</code></pre>
<p><strong>Result</strong></p>
<pre><code>time=&quot;2023-01-09T07:50:56Z&quot; level=debug msg=&quot;debug flag set to true&quot; version=5.3.0-0
time=&quot;2023-01-09T07:51:26Z&quot; level=error msg=&quot;Failed to get API Group-Resources&quot; error=&quot;Get \&quot;https://10.96.0.1:443/api?timeout=32s\&quot;: dial tcp 10.96.0.1:443: i/o timeout&quot; version=5.3.0-0
panic: Get &quot;https://10.96.0.1:443/api?timeout=32s&quot;: dial tcp 10.96.0.1:443: i/o timeout

goroutine 1 [running]:
main.assertNoError(...)
        github.com/crunchydata/postgres-operator/cmd/postgres-operator/main.go:42
main.main()
        github.com/crunchydata/postgres-operator/cmd/postgres-operator/main.go:84 +0x465
</code></pre>
<p>To check the network connection to host I run this command</p>
<pre><code>wget https://10.96.0.1:443/api
</code></pre>
<p><strong>The Result is</strong></p>
<pre><code>--2023-01-09 09:49:30--  https://10.96.0.1/api
Connecting to 10.96.0.1:443... connected.
ERROR: cannot verify 10.96.0.1's certificate, issued by ‘CN=kubernetes’:
  Unable to locally verify the issuer's authority.
To connect to 10.96.0.1 insecurely, use `--no-check-certificate'.
</code></pre>
<p>As you can see it is connected to API</p>
<p><strong>Strange issue might be useful to help me</strong></p>
<p>I run <code>kubectl get pods --all-namespaces</code> and see this output</p>
<pre><code>NAMESPACE           NAME                                                READY   STATUS             RESTARTS         AGE
kube-flannel        kube-flannel-ds-9gmmq                               1/1     Running            0                3d16h
kube-flannel        kube-flannel-ds-rcq8l                               0/1     CrashLoopBackOff   10 (3m15s ago)   34m
kube-flannel        kube-flannel-ds-rqwtj                               0/1     CrashLoopBackOff   10 (2m53s ago)   34m
kube-system         etcd-masterk8s-virtual-machine                      1/1     Running            1 (5d ago)       3d17h
kube-system         kube-apiserver-masterk8s-virtual-machine            1/1     Running            2 (5d ago)       3d17h
kube-system         kube-controller-manager-masterk8s-virtual-machine   1/1     Running            8 (2d ago)       3d17h
kube-system         kube-scheduler-masterk8s-virtual-machine            1/1     Running            7 (5d ago)       3d17h
postgres-operator   pgo-98c6b8888-fz8zj                                 0/1     CrashLoopBackOff   7 (4m59s ago)    20m
</code></pre>
<p>As you can see my two kube-flannel Pods are also in crash loop-back and one is running. I am not sure if this is the main cause of this problem</p>
<p><strong>What I want?</strong>
I want to run the PGO pod successfully with no error.</p>
<p><strong>How you can help me?</strong>
Please help me to find the issue or any other way to get detailed logs. I am not able to find the root cause of this problem because,
If it was network issue then why its connected?
if its something else then how can I find the information?</p>
<p><strong>Update and New errors after apply the fixes:</strong></p>
<pre><code>time=&quot;2023-01-09T11:57:47Z&quot; level=debug msg=&quot;debug flag set to true&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Metrics server is starting to listen&quot; addr=&quot;:8080&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;upgrade checking enabled&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;starting controller runtime manager and will wait for signal to exit&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting server&quot; addr=&quot;[::]:8080&quot; kind=metrics path=/metrics version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=debug msg=&quot;upgrade check issue: namespace not set&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1beta1.PostgresCluster&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.ConfigMap&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.Endpoints&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.PersistentVolumeClaim&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.Secret&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.Service&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.ServiceAccount&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.Deployment&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.StatefulSet&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.Job&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.Role&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.RoleBinding&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.CronJob&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.PodDisruptionBudget&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.Pod&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting EventSource&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster source=&quot;kind source: *v1.StatefulSet&quot; version=5.3.0-0
time=&quot;2023-01-09T11:57:47Z&quot; level=info msg=&quot;Starting Controller&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster version=5.3.0-0
W0109 11:57:48.006419       1 reflector.go:324] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
E0109 11:57:48.006642       1 reflector.go:138] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
time=&quot;2023-01-09T11:57:49Z&quot; level=info msg=&quot;{\&quot;pgo_versions\&quot;:[{\&quot;tag\&quot;:\&quot;v5.1.0\&quot;},{\&quot;tag\&quot;:\&quot;v5.0.5\&quot;},{\&quot;tag\&quot;:\&quot;v5.0.4\&quot;},{\&quot;tag\&quot;:\&quot;v5.0.3\&quot;},{\&quot;tag\&quot;:\&quot;v5.0.2\&quot;},{\&quot;tag\&quot;:\&quot;v5.0.1\&quot;},{\&quot;tag\&quot;:\&quot;v5.0.0\&quot;}]}&quot; X-Crunchy-Client-Metadata=&quot;{\&quot;deployment_id\&quot;:\&quot;288f4766-8617-479b-837f-2ee59ce2049a\&quot;,\&quot;kubernetes_env\&quot;:\&quot;v1.26.0\&quot;,\&quot;pgo_clusters_total\&quot;:0,\&quot;pgo_version\&quot;:\&quot;5.3.0-0\&quot;,\&quot;is_open_shift\&quot;:false}&quot; version=5.3.0-0
W0109 11:57:49.163062       1 reflector.go:324] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
E0109 11:57:49.163119       1 reflector.go:138] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
W0109 11:57:51.404639       1 reflector.go:324] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
E0109 11:57:51.404811       1 reflector.go:138] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
W0109 11:57:54.749751       1 reflector.go:324] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
E0109 11:57:54.750068       1 reflector.go:138] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
W0109 11:58:06.015650       1 reflector.go:324] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
E0109 11:58:06.015710       1 reflector.go:138] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
W0109 11:58:25.355009       1 reflector.go:324] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
E0109 11:58:25.355391       1 reflector.go:138] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
W0109 11:59:10.447123       1 reflector.go:324] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
E0109 11:59:10.447490       1 reflector.go:138] k8s.io/client-go@v0.24.2/tools/cache/reflector.go:167: Failed to watch *v1.PodDisruptionBudget: failed to list *v1.PodDisruptionBudget: poddisruptionbudgets.policy is forbidden: User &quot;system:serviceaccount:postgres-operator:pgo&quot; cannot list resource &quot;poddisruptionbudgets&quot; in API group &quot;policy&quot; at the cluster scope
time=&quot;2023-01-09T11:59:47Z&quot; level=error msg=&quot;Could not wait for Cache to sync&quot; controller=postgrescluster controllerGroup=postgres-operator.crunchydata.com controllerKind=PostgresCluster error=&quot;failed to wait for postgrescluster caches to sync: timed out waiting for cache to be synced&quot; version=5.3.0-0
time=&quot;2023-01-09T11:59:47Z&quot; level=info msg=&quot;Stopping and waiting for non leader election runnables&quot; version=5.3.0-0
time=&quot;2023-01-09T11:59:47Z&quot; level=info msg=&quot;Stopping and waiting for leader election runnables&quot; version=5.3.0-0
time=&quot;2023-01-09T11:59:47Z&quot; level=info msg=&quot;Stopping and waiting for caches&quot; version=5.3.0-0
time=&quot;2023-01-09T11:59:47Z&quot; level=error msg=&quot;failed to get informer from cache&quot; error=&quot;Timeout: failed waiting for *v1.PodDisruptionBudget Informer to sync&quot; version=5.3.0-0
time=&quot;2023-01-09T11:59:47Z&quot; level=error msg=&quot;error received after stop sequence was engaged&quot; error=&quot;context canceled&quot; version=5.3.0-0
time=&quot;2023-01-09T11:59:47Z&quot; level=info msg=&quot;Stopping and waiting for webhooks&quot; version=5.3.0-0
time=&quot;2023-01-09T11:59:47Z&quot; level=info msg=&quot;Wait completed, proceeding to shutdown the manager&quot; version=5.3.0-0
panic: failed to wait for postgrescluster caches to sync: timed out waiting for cache to be synced

goroutine 1 [running]:
main.assertNoError(...)
        github.com/crunchydata/postgres-operator/cmd/postgres-operator/main.go:42
main.main()
        github.com/crunchydata/postgres-operator/cmd/postgres-operator/main.go:118 +0x434
</code></pre>

tauqeerahmad24
<p>If this is a new deployment, I suggest using <a href="https://access.crunchydata.com/documentation/postgres-operator/v5/" rel="nofollow noreferrer">v5</a>.</p>
<p>That said, as PGO manages the networking for Postgres clusters (and as such, manages listen_adresses), there's no reason to modify the listen_addresses configuration parameter. If you need to manage networking or networking access, you can do that by setting the pg_hba config or using <a href="https://kubernetes.io/docs/concepts/services-networking/network-policies/" rel="nofollow noreferrer">NetworkPolicies</a>.</p>
<p>Please go through the <a href="https://github.com/CrunchyData/postgres-operator/issues/2904" rel="nofollow noreferrer">Custom 'listen_addresses' not applied #2904</a> for more information.</p>
<p><strong>CrashLoopBackOff:</strong> Check the pod logs for configuration or deployment issues such as missing dependencies (Like : kubernetes engine doesn't support docker-compose depends-on, so now we are using kubernetes + docker without nginx) and also check for pods being OOM killed and excessive resource usage.</p>
<p>Check for the <a href="https://github.com/projectcalico/calico/issues/3422" rel="nofollow noreferrer">timeout issues</a> and also <a href="https://forum.linuxfoundation.org/discussion/856520/lab-12-7-problem-timeouts-on-dashboard" rel="nofollow noreferrer">lab on timeout problem</a></p>
<pre><code>ERROR: cannot verify 10.96.0.1's certificate, issued by ‘CN=kubernetes’:
  Unable to locally verify the issuer's authority.
To connect to 10.96.0.1 insecurely, use `--no-check-certificate'.
</code></pre>
<p><strong>Try solution for the above Error :</strong>
first, remove ip link flannel.1 on every hosts which has this problem</p>
<p>secondly, delete kube-flannel-ds from k8s</p>
<p>last, recreate kube-flannel-ds from k8s, flannel.1 ip link will recreated and return back good.</p>
<p>(For flannel to work correctly, you must pass <code>--pod-network-cidr=10.244.0.0/16</code> to kubeadm init.(I mean Change CIDR).)</p>
<p><strong>Edit :</strong></p>
<p>Please check similar <a href="https://bugzilla.redhat.com/show_bug.cgi?id=2040136" rel="nofollow noreferrer">issue and solution</a> ,which may help to resolve your issue.</p>
