{"action":"created","issue":{"url":"https://api.github.com/repos/rstudio/sparklyr/issues/1356","repository_url":"https://api.github.com/repos/rstudio/sparklyr","labels_url":"https://api.github.com/repos/rstudio/sparklyr/issues/1356/labels{/name}","comments_url":"https://api.github.com/repos/rstudio/sparklyr/issues/1356/comments","events_url":"https://api.github.com/repos/rstudio/sparklyr/issues/1356/events","html_url":"https://github.com/rstudio/sparklyr/issues/1356","id":305983448,"number":1356,"title":"Can idle sparklyr sessions be cleaned up?","user":{"login":"mattpollock","id":9449081,"avatar_url":"https://avatars2.githubusercontent.com/u/9449081?v=4","gravatar_id":"","url":"https://api.github.com/users/mattpollock","html_url":"https://github.com/mattpollock","followers_url":"https://api.github.com/users/mattpollock/followers","following_url":"https://api.github.com/users/mattpollock/following{/other_user}","gists_url":"https://api.github.com/users/mattpollock/gists{/gist_id}","starred_url":"https://api.github.com/users/mattpollock/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattpollock/subscriptions","organizations_url":"https://api.github.com/users/mattpollock/orgs","repos_url":"https://api.github.com/users/mattpollock/repos","events_url":"https://api.github.com/users/mattpollock/events{/privacy}","received_events_url":"https://api.github.com/users/mattpollock/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-03-16T15:58:11Z","updated_at":"2018-03-27T18:09:10Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"referencing https://stackoverflow.com/q/49033520/1785752.\r\n\r\nSince hadoop clusters tend to be shared resources, forgetful analysts who do not `spark_disconnect(sc)` when finished working cause problems. This tends not to by an issue for pySpark because jupyter enables the [culling of kernels](http://www.spark.tc/limit-notebook-resource-consumption-by-culling-kernels/). However RStudio Server is friendlier and, though it does suspend sessions, will reinstate them when a user comes back. This is generally very useful *except* that session suspend does not terminate a spark process.\r\n\r\nIs it possible to add a timeout argument (if R or the spark session have been idle for x amount of time)? Can a callback be used to trigger Spark sessions to disconnect when RStudio suspends a session?\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/rstudio/sparklyr/issues/comments/376621677","html_url":"https://github.com/rstudio/sparklyr/issues/1356#issuecomment-376621677","issue_url":"https://api.github.com/repos/rstudio/sparklyr/issues/1356","id":376621677,"user":{"login":"mattpollock","id":9449081,"avatar_url":"https://avatars2.githubusercontent.com/u/9449081?v=4","gravatar_id":"","url":"https://api.github.com/users/mattpollock","html_url":"https://github.com/mattpollock","followers_url":"https://api.github.com/users/mattpollock/followers","following_url":"https://api.github.com/users/mattpollock/following{/other_user}","gists_url":"https://api.github.com/users/mattpollock/gists{/gist_id}","starred_url":"https://api.github.com/users/mattpollock/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattpollock/subscriptions","organizations_url":"https://api.github.com/users/mattpollock/orgs","repos_url":"https://api.github.com/users/mattpollock/repos","events_url":"https://api.github.com/users/mattpollock/events{/privacy}","received_events_url":"https://api.github.com/users/mattpollock/received_events","type":"User","site_admin":false},"created_at":"2018-03-27T18:09:10Z","updated_at":"2018-03-27T18:09:10Z","author_association":"CONTRIBUTOR","body":"Thanks, I'll watch that one. We are using dynamic allocation by default, but long running idle connections (order of several days since we are running off an RStudio Server process that lives on a VM that doesn't shut down often) still upsets the administrators."}}
{"actor":{"display_login":"mattpollock"}}