{"action":"created","issue":{"url":"https://api.github.com/repos/rstudio/sparklyr/issues/1742","repository_url":"https://api.github.com/repos/rstudio/sparklyr","labels_url":"https://api.github.com/repos/rstudio/sparklyr/issues/1742/labels{/name}","comments_url":"https://api.github.com/repos/rstudio/sparklyr/issues/1742/comments","events_url":"https://api.github.com/repos/rstudio/sparklyr/issues/1742/events","html_url":"https://github.com/rstudio/sparklyr/issues/1742","id":375913142,"node_id":"MDU6SXNzdWUzNzU5MTMxNDI=","number":1742,"title":"Spark Connection - not loading Hive Metastore in AWS Glue","user":{"login":"preitz07","id":30109005,"node_id":"MDQ6VXNlcjMwMTA5MDA1","avatar_url":"https://avatars1.githubusercontent.com/u/30109005?v=4","gravatar_id":"","url":"https://api.github.com/users/preitz07","html_url":"https://github.com/preitz07","followers_url":"https://api.github.com/users/preitz07/followers","following_url":"https://api.github.com/users/preitz07/following{/other_user}","gists_url":"https://api.github.com/users/preitz07/gists{/gist_id}","starred_url":"https://api.github.com/users/preitz07/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/preitz07/subscriptions","organizations_url":"https://api.github.com/users/preitz07/orgs","repos_url":"https://api.github.com/users/preitz07/repos","events_url":"https://api.github.com/users/preitz07/events{/privacy}","received_events_url":"https://api.github.com/users/preitz07/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"kevinykuo","id":5582151,"node_id":"MDQ6VXNlcjU1ODIxNTE=","avatar_url":"https://avatars3.githubusercontent.com/u/5582151?v=4","gravatar_id":"","url":"https://api.github.com/users/kevinykuo","html_url":"https://github.com/kevinykuo","followers_url":"https://api.github.com/users/kevinykuo/followers","following_url":"https://api.github.com/users/kevinykuo/following{/other_user}","gists_url":"https://api.github.com/users/kevinykuo/gists{/gist_id}","starred_url":"https://api.github.com/users/kevinykuo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kevinykuo/subscriptions","organizations_url":"https://api.github.com/users/kevinykuo/orgs","repos_url":"https://api.github.com/users/kevinykuo/repos","events_url":"https://api.github.com/users/kevinykuo/events{/privacy}","received_events_url":"https://api.github.com/users/kevinykuo/received_events","type":"User","site_admin":false},"assignees":[{"login":"kevinykuo","id":5582151,"node_id":"MDQ6VXNlcjU1ODIxNTE=","avatar_url":"https://avatars3.githubusercontent.com/u/5582151?v=4","gravatar_id":"","url":"https://api.github.com/users/kevinykuo","html_url":"https://github.com/kevinykuo","followers_url":"https://api.github.com/users/kevinykuo/followers","following_url":"https://api.github.com/users/kevinykuo/following{/other_user}","gists_url":"https://api.github.com/users/kevinykuo/gists{/gist_id}","starred_url":"https://api.github.com/users/kevinykuo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kevinykuo/subscriptions","organizations_url":"https://api.github.com/users/kevinykuo/orgs","repos_url":"https://api.github.com/users/kevinykuo/repos","events_url":"https://api.github.com/users/kevinykuo/events{/privacy}","received_events_url":"https://api.github.com/users/kevinykuo/received_events","type":"User","site_admin":false}],"milestone":null,"comments":0,"created_at":"2018-10-31T11:17:19Z","updated_at":"2018-11-02T06:04:11Z","closed_at":null,"author_association":"NONE","body":"We are using R-Studio installed on a master node in AWS EMR. On the same node we have Spark, Hue, Hive, Zeppelin, Yarn etc. Our Hive Metadatastore is AWS Glue and Zeppelin and Hue can interact with it perfectly. We are however not able to create a working connection with sparklyr (working connection means access to the data, the connection is still \"connecting\" to something but no data is visible or accessible). It was working a few months back, but with the latest version it is no longer possible. We tried everything described here: https://github.com/rstudio/sparklyr/issues/1496 but to no avail.\r\n\r\n\r\n```r\r\ndevtools::install_github(\"rstudio/sparklyr\")\r\n\r\nlibrary(sparklyr)\r\nspark_version <- \"2.3.2\"\r\nsc <- spark_connect(master = \"yarn-client\", version=\"2.3.2\")\r\n```\r\n\r\n> utils::sessionInfo()\r\nR version 3.4.1 (2017-06-30)\r\nPlatform: x86_64-redhat-linux-gnu (64-bit)\r\nRunning under: Amazon Linux AMI 2018.03\r\nMatrix products: default\r\nBLAS/LAPACK: /usr/lib64/R/lib/libRblas.so\r\nlocale:\r\n [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   \r\n [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            \r\n[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C    \r\nattached base packages:\r\n[1] stats     graphics  grDevices utils     datasets  methods   base     \r\nother attached packages:\r\n[1] dplyr_0.7.7         sparklyr_0.9.2.9000 usethis_1.4.0       devtools_2.0.1     \r\nloaded via a namespace (and not attached):\r\n [1] tidyselect_0.2.5  forge_0.1.0       remotes_2.0.1     purrr_0.2.5       lattice_0.20-35   htmltools_0.3.6   yaml_2.2.0        base64enc_0.1-3  \r\n [9] rlang_0.3.0.1     pkgbuild_1.0.2    later_0.7.5       pillar_1.3.0      glue_1.3.0        withr_2.1.2       DBI_1.0.0         dbplyr_1.2.2     \r\n[17] sessioninfo_1.1.0 bindrcpp_0.2.2    bindr_0.1.1       htmlwidgets_1.3   memoise_1.1.0     callr_3.0.0       httpuv_1.4.5      ps_1.2.0         \r\n[25] parallel_3.4.1    curl_3.2          broom_0.5.0       r2d3_0.2.2        Rcpp_0.12.19      xtable_1.8-3      openssl_1.0.2     promises_1.0.1   \r\n[33] backports_1.1.2   desc_1.2.0        pkgload_1.0.1     jsonlite_1.5      config_0.3        mime_0.6          fs_1.2.6          digest_0.6.18    \r\n[41] processx_3.2.0    shiny_1.1.0       grid_3.4.1        rprojroot_1.3-2   cli_1.0.1         tools_3.4.1       magrittr_1.5      lazyeval_0.2.1   \r\n[49] tibble_1.4.2      crayon_1.3.4      tidyr_0.8.1       pkgconfig_2.0.2   prettyunits_1.0.2 assertthat_0.2.0  httr_1.3.1        rstudioapi_0.8   \r\n[57] R6_2.3.0          nlme_3.1-131      compiler_3.4.1   \r\n"},"comment":{"url":"https://api.github.com/repos/rstudio/sparklyr/issues/comments/435280315","html_url":"https://github.com/rstudio/sparklyr/issues/1742#issuecomment-435280315","issue_url":"https://api.github.com/repos/rstudio/sparklyr/issues/1742","id":435280315,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNTI4MDMxNQ==","user":{"login":"javierluraschi","id":3478847,"node_id":"MDQ6VXNlcjM0Nzg4NDc=","avatar_url":"https://avatars1.githubusercontent.com/u/3478847?v=4","gravatar_id":"","url":"https://api.github.com/users/javierluraschi","html_url":"https://github.com/javierluraschi","followers_url":"https://api.github.com/users/javierluraschi/followers","following_url":"https://api.github.com/users/javierluraschi/following{/other_user}","gists_url":"https://api.github.com/users/javierluraschi/gists{/gist_id}","starred_url":"https://api.github.com/users/javierluraschi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/javierluraschi/subscriptions","organizations_url":"https://api.github.com/users/javierluraschi/orgs","repos_url":"https://api.github.com/users/javierluraschi/repos","events_url":"https://api.github.com/users/javierluraschi/events{/privacy}","received_events_url":"https://api.github.com/users/javierluraschi/received_events","type":"User","site_admin":false},"created_at":"2018-11-02T06:04:11Z","updated_at":"2018-11-02T06:04:11Z","author_association":"MEMBER","body":"@preitz07 could you share the exception that you are hitting? Which is the latest version of `sparklyr` where this was working correctly?"}}
{"actor":{"display_login":"javierluraschi"}}