{"action":"created","issue":{"url":"https://api.github.com/repos/singularityware/singularity/issues/1476","repository_url":"https://api.github.com/repos/singularityware/singularity","labels_url":"https://api.github.com/repos/singularityware/singularity/issues/1476/labels{/name}","comments_url":"https://api.github.com/repos/singularityware/singularity/issues/1476/comments","events_url":"https://api.github.com/repos/singularityware/singularity/issues/1476/events","html_url":"https://github.com/singularityware/singularity/issues/1476","id":316264917,"number":1476,"title":"Cannot create image when setting SINGULARITY_TMPDIR and SINGULARITY_CACHEDIR to BeeGFS network filesystem.","user":{"login":"titansmc","id":8192096,"avatar_url":"https://avatars2.githubusercontent.com/u/8192096?v=4","gravatar_id":"","url":"https://api.github.com/users/titansmc","html_url":"https://github.com/titansmc","followers_url":"https://api.github.com/users/titansmc/followers","following_url":"https://api.github.com/users/titansmc/following{/other_user}","gists_url":"https://api.github.com/users/titansmc/gists{/gist_id}","starred_url":"https://api.github.com/users/titansmc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/titansmc/subscriptions","organizations_url":"https://api.github.com/users/titansmc/orgs","repos_url":"https://api.github.com/users/titansmc/repos","events_url":"https://api.github.com/users/titansmc/events{/privacy}","received_events_url":"https://api.github.com/users/titansmc/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2018-04-20T13:20:07Z","updated_at":"2018-04-23T13:12:47Z","closed_at":null,"author_association":"NONE","body":"## Version of Singularity:\r\n\r\n   [moscardo@login ~]# singularity --version\r\n    2.4.6-dist\r\n\r\n### Expected behavior\r\nJust create the expected image (it needs root access ) with the following command\r\n\r\n    SINGULARITY_TMPDIR=/scratch/singularity/tmp/ SINGULARITY_CACHEDIR=/scratch//singularity/cache singularity --debug build star.img docker://machalen/star\r\n\r\nIt works if I set up  SINGULARITY_TMPDIR and SINGULARITY_CACHEDIR to local filesystems but they fail to build when I set them to BeeGFS like /scratch . With NFS it works too.\r\n\r\n### Actual behavior\r\n````\r\n[moscardo@login ~]$ SINGULARITY_TMPDIR=/scratch/singularity/tmp/ SINGULARITY_CACHEDIR=/scratch//singularity/cache singularity --debug build star.img docker://machalen/star\r\nEnabling debugging\r\nEnding argument loop\r\nSingularity version: 2.4.6-dist\r\nExec'ing: /usr/libexec/singularity/cli/build.exec\r\nVERBOSE2 SINGULARITY_COMMAND_ASIS found as False\r\nVERBOSE2 SINGULARITY_ROOTFS found as /scratch/singularity/tmp//.singularity-build.rBAK8x\r\nVERBOSE2 SINGULARITY_METADATA_FOLDER found as /scratch/singularity/tmp//.singularity-build.rBAK8x/.singularity.d\r\nVERBOSE2 SINGULARITY_FIX_PERMS found as False\r\nVERBOSE2 SINGULARITY_COLORIZE not defined (None)\r\nVERBOSE2 SINGULARITY_DISABLE_CACHE found as False\r\nVERBOSE2 SINGULARITY_CACHEDIR found as /scratch//singularity/cache\r\nVERBOSE2 REGISTRY not defined (None)\r\nVERBOSE2 NAMESPACE not defined (None)\r\nVERBOSE2 SINGULARITY_DOCKER_ARCHITECTURE found as amd64\r\nVERBOSE2 SINGULARITY_DOCKER_OS found as linux\r\nVERBOSE2 SINGULARITY_ENVIRONMENT found as /scratch/singularity/tmp//.singularity-build.rBAK8x/.singularity.d/env/90-environment.sh\r\nVERBOSE2 SINGULARITY_RUNSCRIPT found as /scratch/singularity/tmp//.singularity-build.rBAK8x/singularity\r\nVERBOSE2 SINGULARITY_TESTFILE found as /scratch/singularity/tmp//.singularity-build.rBAK8x/.singularity.d/test\r\nVERBOSE2 SINGULARITY_DEFFILE found as /scratch/singularity/tmp//.singularity-build.rBAK8x/.singularity.d/Singularity\r\nVERBOSE2 SINGULARITY_HELPFILE found as /scratch/singularity/tmp//.singularity-build.rBAK8x/.singularity.d/runscript.help\r\nVERBOSE2 SINGULARITY_ENVBASE found as /scratch/singularity/tmp//.singularity-build.rBAK8x/.singularity.d/env\r\nVERBOSE2 SINGULARITY_LABELFILE found as /scratch/singularity/tmp//.singularity-build.rBAK8x/.singularity.d/labels.json\r\nVERBOSE2 SINGULARITY_INCLUDECMD found as False\r\nVERBOSE2 SINGULARITY_NOHTTPS found as False\r\nVERBOSE2 SINGULARITY_PULLFOLDER found as /home/moscardo\r\nVERBOSE2 SHUB_NAMEBYHASH not defined (None)\r\nVERBOSE2 SHUB_NAMEBYCOMMIT not defined (None)\r\nVERBOSE2 SHUB_CONTAINERNAME not defined (None)\r\nVERBOSE2 SINGULARITY_CONTENTS found as /tmp/.singularity-layers.IKzdqBYL\r\nVERBOSE2 SINGULARITY_PYTHREADS found as 9\r\nVERBOSE2 SINGULARITY_CONTAINER found as docker://machalen/star\r\nDEBUG Found uri docker://\r\nDEBUG \r\n*** STARTING DOCKER IMPORT PYTHON  ****\r\nDEBUG Docker layers and metadata will be written to: /tmp/.singularity-layers.IKzdqBYL\r\nVERBOSE2 SINGULARITY_DOCKER_USERNAME not defined (None)\r\nDEBUG Starting Docker IMPORT, includes env, runscript, and metadata.\r\nVERBOSE Docker image: machalen/star\r\nVERBOSE2 Specified Docker ENTRYPOINT as %runscript.\r\nDEBUG Headers found: Content-Type,Accept\r\nVERBOSE Registry: index.docker.io\r\nVERBOSE Namespace: machalen\r\nVERBOSE Repo Name: star\r\nVERBOSE Repo Tag: latest\r\nVERBOSE Version: None\r\nVERBOSE Obtaining tags: https://index.docker.io/v2/machalen/star/tags/list\r\nDEBUG GET https://index.docker.io/v2/machalen/star/tags/list\r\nDEBUG Http Error with code 401\r\nDEBUG GET https://auth.docker.io/token?service=registry.docker.io&expires_in=9000&scope=repository:machalen/star:pull\r\nDEBUG Headers found: Content-Type,Authorization,Accept\r\nVERBOSE3 Response on obtaining token is None.\r\nDocker image path: index.docker.io/machalen/star:latest\r\nDEBUG Updating manifests.\r\nDEBUG MANIFEST (Primary): not found, making initial call.\r\nVERBOSE Obtaining manifest: https://index.docker.io/v2/machalen/star/manifests/latest\r\nDEBUG GET https://index.docker.io/v2/machalen/star/manifests/latest\r\nDEBUG MANIFEST (Metadata): not found, making initial call.\r\nVERBOSE Obtaining manifest: https://index.docker.io/v2/machalen/star/manifests/latest\r\nDEBUG GET https://index.docker.io/v2/machalen/star/manifests/latest\r\nDEBUG Image manifest version 2.2 found.\r\nDEBUG Adding digest sha256:1be7f2b886e89a58e59c4e685fcc5905a26ddef3201f290b96f1eff7d778e122\r\nDEBUG Adding digest sha256:6fbc4a21b806838b63b774b338c6ad19d696a9e655f50b4e358cc4006c3baa79\r\nDEBUG Adding digest sha256:c71a6f8e13782fed125f2247931c3eb20cc0e6428a5d79edb546f1f1405f0e49\r\nDEBUG Adding digest sha256:4be3072e5a37392e32f632bb234c0b461ff5675ab7e362afad6359fbd36884af\r\nDEBUG Adding digest sha256:06c6d2f5970057aef3aef6da60f0fde280db1c077f0cd88ca33ec1a70a9c7b58\r\nDEBUG Adding digest sha256:3741549fd861581a127b722ef660a749ca81605e222ada192d0b77b7f5ed125c\r\nDEBUG Adding digest sha256:e1825c76c885492b2b7a94c540054ef1bc998dddc01129dfc678135d4d5533db\r\nDEBUG Adding digest sha256:2c4f8b76306cc612b97f8587e73aee69cbd857ec3665106098f2f44d579d21c2\r\nDEBUG Adding digest sha256:1c58d960038bbc023e20fc982a04de7fb801aa79585c66b3eda62eae4a219c68\r\nDEBUG Adding digest sha256:1236c32f280781573087f49b84e40a0c0ceadeacd8ce1da15077d56c28fdaccd\r\nDEBUG Adding digest sha256:1a3c6a91c272f210f261d619d3894c2c26fb856804a8a4f6911bd3a30408d7e9\r\nDEBUG Adding digest sha256:3814051a201b86afc8fd03c6de736ed999fd48a85075a206a10db0291946979d\r\nDEBUG Adding digest sha256:1b4e60a384e5e4e667a9401da372a71936f7752158fab800b059997943defe43\r\nDEBUG Adding digest sha256:c53773e3cf413bb830235815a0b15a417f488020069e8a5fd238ea3ebb8d6b46\r\nDEBUG Adding digest sha256:15dc8473bdc90054861159f56e295f6321256c1e7f304b2e6e0074e08d939cdd\r\nDEBUG Adding digest sha256:ba37efa138912e26dc61967c10d4f2a10dceb8f8d7120f4dfef8cc1804105847\r\nCache folder set to /scratch/singularity/cache/docker\r\nDEBUG Using 9 workers for multiprocess.\r\nVERBOSE3 Found Docker command (Entrypoint) None\r\nVERBOSE3 Found Docker command (Cmd) [u'/bin/bash']\r\nVERBOSE3 Adding Docker CMD as Singularity runscript...\r\nVERBOSE3 Found Docker command (Env) [u'PATH=/usr/local/STAR/source:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin/samtools-1.5']\r\nVERBOSE3 Found Docker container environment!\r\nVERBOSE3 Adding Docker environment to metadata tar\r\nDEBUG Found template for tarinfo\r\nVERBOSE3 Found Docker command (Labels) None\r\nVERBOSE3 Adding Docker runscript to metadata tar\r\nDEBUG Found template for tarinfo\r\nDEBUG metadata file /scratch/singularity/cache/metadata/sha256:dd389502ec5ac0759013983a56ea1f6202643a49f065076b02f4a1bb22e6794b.tar.gz already exists will over-write.\r\nVERBOSE2 Tar file with Docker env and labels: /scratch/singularity/cache/metadata/sha256:dd389502ec5ac0759013983a56ea1f6202643a49f065076b02f4a1bb22e6794b.tar.gz\r\nVERBOSE3 Writing Docker layers files to /tmp/.singularity-layers.IKzdqBYL\r\nVERBOSE2 Writing file /tmp/.singularity-layers.IKzdqBYL with mode w.\r\nVERBOSE2 Writing file /tmp/.singularity-layers.IKzdqBYL with mode a.\r\nDEBUG *** FINISHING DOCKER IMPORT PYTHON PORTION ****\r\nImporting: base Singularity environment\r\nImporting: /scratch/singularity/cache/docker/sha256:1be7f2b886e89a58e59c4e685fcc5905a26ddef3201f290b96f1eff7d778e122.tar.gz\r\nImporting: /scratch/singularity/cache/docker/sha256:6fbc4a21b806838b63b774b338c6ad19d696a9e655f50b4e358cc4006c3baa79.tar.gz\r\nImporting: /scratch/singularity/cache/docker/sha256:c71a6f8e13782fed125f2247931c3eb20cc0e6428a5d79edb546f1f1405f0e49.tar.gz\r\nImporting: /scratch/singularity/cache/docker/sha256:4be3072e5a37392e32f632bb234c0b461ff5675ab7e362afad6359fbd36884af.tar.gz\r\nImporting: /scratch/singularity/cache/docker/sha256:06c6d2f5970057aef3aef6da60f0fde280db1c077f0cd88ca33ec1a70a9c7b58.tar.gz\r\nImporting: /scratch/singularity/cache/docker/sha256:3741549fd861581a127b722ef660a749ca81605e222ada192d0b77b7f5ed125c.tar.gz\r\nImporting: /scratch/singularity/cache/docker/sha256:e1825c76c885492b2b7a94c540054ef1bc998dddc01129dfc678135d4d5533db.tar.gz\r\ntar: etc/rc2.d/.wh.S01ondemand: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc2.d/.wh.S01rc.local: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc3.d/.wh.S01ondemand: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc3.d/.wh.S01rc.local: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc4.d/.wh.S01ondemand: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc4.d/.wh.S01rc.local: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc5.d/.wh.S01ondemand: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc5.d/.wh.S01rc.local: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc6.d/.wh.K03umountfs: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc6.d/.wh.K04umountroot: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rc6.d/.wh.K05reboot: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rcS.d/.wh.S03mountnfs.sh: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rcS.d/.wh.S06mountnfs-bootclean.sh: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: etc/rcS.d/.wh.S09bootmisc.sh: Cannot hard link to `etc/rc0.d/.wh.K03umountfs': Operation not permitted\r\ntar: Exiting with failure status due to previous errors\r\nCleaning up...\r\n````\r\n\r\n### Steps to reproduce behavior\r\n\r\n\r\n    SINGULARITY_TMPDIR=/scratch/singularity/tmp/ SINGULARITY_CACHEDIR=/scratch//singularity/cache singularity --debug build star.img docker://machalen/star\r\n"},"comment":{"url":"https://api.github.com/repos/singularityware/singularity/issues/comments/383570046","html_url":"https://github.com/singularityware/singularity/issues/1476#issuecomment-383570046","issue_url":"https://api.github.com/repos/singularityware/singularity/issues/1476","id":383570046,"user":{"login":"dctrud","id":4522799,"avatar_url":"https://avatars1.githubusercontent.com/u/4522799?v=4","gravatar_id":"","url":"https://api.github.com/users/dctrud","html_url":"https://github.com/dctrud","followers_url":"https://api.github.com/users/dctrud/followers","following_url":"https://api.github.com/users/dctrud/following{/other_user}","gists_url":"https://api.github.com/users/dctrud/gists{/gist_id}","starred_url":"https://api.github.com/users/dctrud/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dctrud/subscriptions","organizations_url":"https://api.github.com/users/dctrud/orgs","repos_url":"https://api.github.com/users/dctrud/repos","events_url":"https://api.github.com/users/dctrud/events{/privacy}","received_events_url":"https://api.github.com/users/dctrud/received_events","type":"User","site_admin":false},"created_at":"2018-04-23T13:12:47Z","updated_at":"2018-04-23T13:12:47Z","author_association":"CONTRIBUTOR","body":"Hi @titansmc - it looks like this later error on the 2.5 branch is due to an extra '/' creeping into the path of the rootfs where Singularity is constructing the container, causing an extraction safety check to fail. It's not BeeGFS related.\r\n\r\nCould you try without the trailing slash on your TMPDIR  - i.e. `/scratch/singularity/tmp` not `/scratch/singularity/tmp/` ? I'll look at getting a fix in for this."}}
{"actor":{"display_login":"dctrud"}}