<p>I have a Kubernetes v1.9.3 (no OpenShift) cluster I'd like to manage with ManageIQ (gaprindashvili-3 running as a Docker container).</p>

<p>I prepared the k8s cluster to interact with ManageIQ following <a href="http://manageiq.org/docs/guides/providers/kubernetes" rel="nofollow noreferrer">these instructions</a>. Notice that I performed the steps listed in the last section only (<strong>Prepare cluster for use with ManageIQ</strong>), as the previous ones were for setting up a k8s cluster and I already had a running one.</p>

<p>I successfully added the k8s container provider to ManageIQ, but the dashboard reports nothing: 0 nodes, 0 pods, 0 services, etc..., while I do have nodes, services and running pods on the cluster. I looked at the content of <code>/var/log/evm.log</code> of ManageIQ and found this error:</p>

<pre><code>[----] E, [2018-06-21T10:06:40.397410 #13333:6bc9e80] ERROR – : [KubeException]: events is forbidden: User “system:serviceaccount:management-infra:management-admin” cannot list events at the cluster scope: clusterrole.rbac.authorization.k8s.io “cluster-reader” not found Method:[block in method_missing]
</code></pre>

<p>So the ClusterRole <code>cluster-reader</code> was not defined in the cluster. I double checked with <code>kubectl get clusterrole cluster-reader</code> and it confirmed that <code>cluster-reader</code> was missing. </p>

<p>As a solution, I tried to create <code>cluster-reader</code> manually. I could not find any reference of it in the k8s doc, while it is mentioned in the OpenShift docs. So I looked at how <code>cluster-reader</code> was defined in OpenShift v3.9. Its definition changes across different OpenShift versions, I picked 3.9 as it is based on k8s v1.9 which is the one I'm using. So here's what I found in the OpenShift 3.9 doc:</p>

<pre><code>Name:       cluster-reader
Labels:     &lt;none&gt;
Annotations:    authorization.openshift.io/system-only=true
        rbac.authorization.kubernetes.io/autoupdate=true
PolicyRule:
  Resources                         Non-Resource URLs   Resource Names  Verbs
  ---------                         -----------------   --------------  -----
                                               [*]         []           [get]
  apiservices.apiregistration.k8s.io           []          []           [get list watch]
  apiservices.apiregistration.k8s.io/status    []          []           [get list watch]
  appliedclusterresourcequotas                 []          []           [get list watch]
</code></pre>

<p>I wrote the following yaml definition to create an equivalent ClusterRole in my cluster:</p>

<pre><code>apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata: 
  name: cluster-reader
rules:
- apiGroups: ["apiregistration"]
  resources: ["apiservices.apiregistration.k8s.io", "apiservices.apiregistration.k8s.io/status"]
  verbs: ["get", "list", "watch"]
- nonResourceURLs: ["*"]
  verbs: ["get"]
</code></pre>

<p>I didn't include <code>appliedclusterresourcequotas</code> among the monitored resources because it's my understanding that is an OpenShift-only resource (but I may be mistaken).</p>

<p>I deleted the old k8s container provider on ManageIQ and created a new one after having created <code>cluster-reader</code>, but nothing changed, the dashboard still displays nothing (0 nodes, 0 pods, etc...). I looked at the content of <code>/var/log/evm.log</code> in ManageIQ and this time these errors were reported:</p>

<pre><code>[----] E, [2018-06-22T11:15:39.041903 #2942:7e5e1e0] ERROR -- : MIQ(ManageIQ::Providers::Kubernetes::ContainerManager::EventCatcher::Runner#start_event_monitor) EMS [kubernetes-01] as [] Event Monitor Thread aborted because [events is forbidden: User "system:serviceaccount:management-infra:management-admin" cannot list events at the cluster scope]
[----] E, [2018-06-22T11:15:39.042455 #2942:7e5e1e0] ERROR -- : [KubeException]: events is forbidden: User "system:serviceaccount:management-infra:management-admin" cannot list events at the cluster scope  Method:[block in method_missing]
</code></pre>

<p>So what am I doing wrong? How can I fix this problem? 
If it can be of any use, <a href="https://github.com/matte21/cloud-infrastructure-project-work/blob/master/k8s-cluster-miq-setup/k8-miq-objects.yaml" rel="nofollow noreferrer">here</a> you can find the whole .yaml file I'm using to set up the k8s cluster to interact with ManageIQ (all the required namespaces, service accounts, cluster role bindings are present as well).</p>

<p>For the <code>ClusterRole</code> to take effect it must be bound to the group <code>management-infra</code> or user <code>management-admin</code>.</p>

<p>Example of creating group binding:</p>

<pre><code>kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-cluster-state
subjects:
- kind: Group
  name: management-infra
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: cluster-reader
  apiGroup: rbac.authorization.k8s.io
</code></pre>

<p>After applying this file changes will take place immediately. No need to restart cluster.</p>

<p>See more information <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/#rolebinding-and-clusterrolebinding" rel="nofollow noreferrer">here</a>.</p>
