{"action":"closed","issue":{"url":"https://api.github.com/repos/Azure/doAzureParallel/issues/324","repository_url":"https://api.github.com/repos/Azure/doAzureParallel","labels_url":"https://api.github.com/repos/Azure/doAzureParallel/issues/324/labels{/name}","comments_url":"https://api.github.com/repos/Azure/doAzureParallel/issues/324/comments","events_url":"https://api.github.com/repos/Azure/doAzureParallel/issues/324/events","html_url":"https://github.com/Azure/doAzureParallel/issues/324","id":379789408,"node_id":"MDU6SXNzdWUzNzk3ODk0MDg=","number":324,"title":"libpaths duplicated twice","user":{"login":"simon-tarr","id":27423094,"node_id":"MDQ6VXNlcjI3NDIzMDk0","avatar_url":"https://avatars3.githubusercontent.com/u/27423094?v=4","gravatar_id":"","url":"https://api.github.com/users/simon-tarr","html_url":"https://github.com/simon-tarr","followers_url":"https://api.github.com/users/simon-tarr/followers","following_url":"https://api.github.com/users/simon-tarr/following{/other_user}","gists_url":"https://api.github.com/users/simon-tarr/gists{/gist_id}","starred_url":"https://api.github.com/users/simon-tarr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/simon-tarr/subscriptions","organizations_url":"https://api.github.com/users/simon-tarr/orgs","repos_url":"https://api.github.com/users/simon-tarr/repos","events_url":"https://api.github.com/users/simon-tarr/events{/privacy}","received_events_url":"https://api.github.com/users/simon-tarr/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-11-12T14:00:44Z","updated_at":"2018-11-14T15:19:21Z","closed_at":"2018-11-14T15:19:21Z","author_association":"NONE","body":"I've been having issues with GitHub limiting the number of API calls to the service, preventing me from installing GitHub packages on every iteration of a foreach loop within doAzureParallel (see #323).\r\n\r\nIn an attempt to sidestep this issue, I decided to create a custom Docker image, using the documentation provided [here](https://github.com/Azure/doAzureParallel/blob/2b8f388dc476a3a51aa71c3ab1165d080ad08fe8/docs/33-building-containers.md). I have been successful in creating an image and I can successfully load the GitHub package required for my analysis locally. I have also been successful in pushing it to Docker hub (it can be found [here](https://hub.docker.com/r/arcalis/nichemapr/)) and getting a cluster to boot with my image.\r\n\r\nThe problem arises, however, when I attempt to load the same GitHub-installed packge into my session on Azure (using `library(NicheMapR)`). If I use a simple example, just to load the package:\r\n\r\n```\r\n# Generate cluster config file with custom docker image\r\n    clusterConfig <- list(\r\n\r\n      \"name\" = \"test_cluster\r\n      \"vmSize\" = \"Standard_E64s_v3\",\r\n      \"maxTasksPerNode\" = 64,\r\n      \"poolSize\" = list(\r\n        \"dedicatedNodes\" = list(\r\n          \"min\" = 0,\r\n          \"max\" = 0\r\n        ),\r\n        \"lowPriorityNodes\" = list(\r\n          \"min\" = 8,\r\n          \"max\" = 8\r\n        ),\r\n        \"autoscaleFormula\" = \"QUEUE\"\r\n      ),\r\n      \"containerImage\" = \"arcalis/nichemapr\",\r\n      \"rPackages\" = list(\r\n        \"cran\" = list(),\r\n        \"github\" = list(),\r\n        \"bioconductor\" = list()\r\n      ),\r\n      \"commandLine\" = list()\r\n    )\r\n\r\n    cluster <<- doAzureParallel::makeCluster(clusterConfig, resourceFiles = resource_files)\r\n\r\ntest <- foreach(i = 1:5, .errorhandling = 'pass') %dopar% {\r\n\r\n  library(NicheMapR)\r\n  sessionInfo()\r\n  \r\n  }\r\n```\r\n\r\nI get the following error for each iteration of the loop: \r\n```\r\n$`1`\r\n<simpleError: package or namespace load failed for â€˜NicheMapRâ€™:\r\n .onLoad failed in loadNamespace() for 'NicheMapR', details:\r\n  call: dyn.load(micro_lib)\r\n  error: unable to load shared object '/mnt/batch/tasks/workitems/job20181112134610/job-1/jobpreparation/wd/mnt/batch/tasks/workitems/job20181112134610/job-1/jobpreparation/wd/NicheMapR/libs/linux/MICROCLIMATE.so':\r\n  /mnt/batch/tasks/workitems/job20181112134610/job-1/jobpreparation/wd/mnt/batch/tasks/workitems/job20181112134610/job-1/jobpreparation/wd/NicheMapR/libs/linux/MICROCLIMATE.so: cannot open shared object file: No such file or directory>\r\n\r\n```\r\n\r\nYou'll notice that the path string` /mnt/batch/tasks/workitems/job20181112134610/job-1/jobpreparation/wd/` is repeated twice. I've scoured the NicheMapR package code for many hours in my forked copy (simon-tarr/nichemapr@libpath-updates) and can't see anywhere the first entry in .libpaths is duplicated twice, which would result in the above duplication.\r\n\r\nIt's probably a NicheMapR issue, in which case, I obviously expect no support. However, I would be grateful if you could share any information from the doAzureParallel side of things which might shed some light on what's going on here.\r\n\r\nThanks,\r\nSimon\r\n\r\n"}}
{"actor":{"display_login":"simon-tarr"}}