<p>I have spark (3.0.1), livy (0.8.0) and Jupyterhub (sparkmagic) running on K8S in specific namespace, Kubernetes master is used as a resource manager.</p>
<p>When trying to create pyspark session in Jupyterhub's notebook I get the error:</p>
<blockquote>
<p>22/02/04 12:09:16 WARN InteractiveSession: Failed to stop RSCDriver.
Killing it... 22/02/04 12:09:18 WARN InteractiveSession: Error
stopping session 2.
io.fabric8.kubernetes.client.KubernetesClientException: Failure
executing: GET at:
<a href="https://kubernetes.default.svc.cluster.local/api/v1/pods?labelSelector=spark-app-tag%2Cspark-role%3Ddriver%2Cspark-app-selector" rel="nofollow noreferrer">https://kubernetes.default.svc.cluster.local/api/v1/pods?labelSelector=spark-app-tag%2Cspark-role%3Ddriver%2Cspark-app-selector</a>.
Message: Forbidden!Configured service account doesn't have access.
Service account may have been revoked. pods is forbidden: User
&quot;system:serviceaccount:namespace:livy-acc&quot; cannot list resource
&quot;pods&quot; in API group &quot;&quot; at the cluster scope.</p>
</blockquote>
<p>This error states that livy's Kubernetes client tries to list all the pods clusterwide but lacking permissions to do that.</p>
<p>Is it possible to restrict/limit the livy to operate in a certain namespace on Kubernetes, as the alternative of giving away cluster role is not an option due some security concerns.</p>

Artyom Rebrov
<p>During code examination of livy 0.8.0 from this repo: <a href="https://github.com/jahstreet/incubator-livy.git" rel="nofollow noreferrer">https://github.com/jahstreet/incubator-livy.git</a> --branch merge/first I have discovered some undocumented livy.server.kubernetes.* properties that can be used to configure how livy runs on K8S.</p>
<p>For K8S namespace restriction following properties can be used:</p>
<pre><code># Comma-separated list of the Kubernetes namespaces to allow for applications creation.
# All namespaces are allowed if empty
livy.server.kubernetes.allowedNamespaces = namespace

# Kubernetes client default namespace
livy.server.kubernetes.defaultNamespace = namespace
</code></pre>
