{"action":"created","issue":{"url":"https://api.github.com/repos/nextflow-io/nextflow/issues/676","repository_url":"https://api.github.com/repos/nextflow-io/nextflow","labels_url":"https://api.github.com/repos/nextflow-io/nextflow/issues/676/labels{/name}","comments_url":"https://api.github.com/repos/nextflow-io/nextflow/issues/676/comments","events_url":"https://api.github.com/repos/nextflow-io/nextflow/issues/676/events","html_url":"https://github.com/nextflow-io/nextflow/issues/676","id":319628958,"number":676,"title":"NF does not see files in PVC on k8s cluster","user":{"login":"wikiselev","id":2342606,"avatar_url":"https://avatars2.githubusercontent.com/u/2342606?v=4","gravatar_id":"","url":"https://api.github.com/users/wikiselev","html_url":"https://github.com/wikiselev","followers_url":"https://api.github.com/users/wikiselev/followers","following_url":"https://api.github.com/users/wikiselev/following{/other_user}","gists_url":"https://api.github.com/users/wikiselev/gists{/gist_id}","starred_url":"https://api.github.com/users/wikiselev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wikiselev/subscriptions","organizations_url":"https://api.github.com/users/wikiselev/orgs","repos_url":"https://api.github.com/users/wikiselev/repos","events_url":"https://api.github.com/users/wikiselev/events{/privacy}","received_events_url":"https://api.github.com/users/wikiselev/received_events","type":"User","site_admin":false},"labels":[{"id":921273768,"url":"https://api.github.com/repos/nextflow-io/nextflow/labels/triage/needs-information","name":"triage/needs-information","color":"958ad8","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":13,"created_at":"2018-05-02T16:42:42Z","updated_at":"2018-05-03T11:37:05Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Hi Paolo,\r\n\r\nWe created the following persistent volume claim in our k8s cluster:\r\n```\r\napiVersion: v1\r\nkind: PersistentVolumeClaim\r\nmetadata:\r\n  name: testpvc\r\nspec:\r\n  accessModes:\r\n  - ReadWriteMany\r\n  resources:\r\n    requests:\r\n      storage: 50Gi\r\n```\r\n\r\nWe mounted a gluster volume to it and put some data inside:\r\n```\r\nubuntu@vlad-k8s-test-k8s-master-nf-1:~$ tree /mnt/gluster/pvc-319c8c17-3ca6-11e8-89b1-fa163e31bb09\r\n/mnt/gluster/pvc-319c8c17-3ca6-11e8-89b1-fa163e31bb09\r\n`-- data\r\n    |-- 22028_2#111_1.fastq.gz\r\n    |-- 22028_2#111_2.fastq.gz\r\n    |-- 22028_2#11_1.fastq.gz\r\n    |-- 22028_2#11_2.fastq.gz\r\n    |-- 22028_2#118_1.fastq.gz\r\n    `-- 22028_2#118_2.fastq.gz\r\n\r\n1 directory, 6 files\r\n```\r\n\r\nThen I ran the nf-core/rnaseq pipeline, which by default consumes `fastq.gz` files in the `data` folder and got this:\r\n```\r\nubuntu@k8s-master:~/kubespray$ ./nextflow kuberun nf-core/rnaseq --genome GRCh37 -v testpvc:/mnt/gluster/\r\nPod started: nice-jang\r\nN E X T F L O W  ~  version 0.28.0\r\nPulling nf-core/rnaseq ...\r\n downloaded from https://github.com/nf-core/rnaseq.git\r\nLaunching `nf-core/rnaseq` [nice-jang] - revision: 98ffbce439 [master]\r\n===================================\r\n nfcore/rnaseq  ~  version 1.5dev\r\n===================================\r\nERROR ~ Cannot find any reads matching: data/*{1,2}.fastq.gz\r\nNB: Path needs to be enclosed in quotes!\r\nNB: Path requires at least one * wildcard!\r\nIf this is single-end data, please specify --singleEnd on the command line.\r\n\r\n -- Check '.nextflow.log' file for details\r\n```\r\n\r\nThen I looked again in the mounted volume:\r\n```\r\nubuntu@vlad-k8s-test-k8s-master-nf-1:~$ tree /mnt/gluster/pvc-319c8c17-3ca6-11e8-89b1-fa163e31bb09\r\n/mnt/gluster/pvc-319c8c17-3ca6-11e8-89b1-fa163e31bb09\r\n|-- data\r\n|   |-- 22028_2#111_1.fastq.gz\r\n|   |-- 22028_2#111_2.fastq.gz\r\n|   |-- 22028_2#11_1.fastq.gz\r\n|   |-- 22028_2#11_2.fastq.gz\r\n|   |-- 22028_2#118_1.fastq.gz\r\n|   `-- 22028_2#118_2.fastq.gz\r\n|-- projects\r\n|   `-- nf-core\r\n|       `-- rnaseq\r\n|           |-- assets\r\n|           |   |-- biotypes_header.txt\r\n|           |   |-- email_template.html\r\n|           |   |-- email_template.txt\r\n|           |   |-- heatmap_header.txt\r\n|           |   |-- mdsplot_header.txt\r\n|           |   |-- nfcore-rnaseq_logo.png\r\n|           |   |-- NGI_logo.png\r\n|           |   |-- SciLifeLab_logo.png\r\n|           |   |-- sendmail_template.txt\r\n|           |   `-- where_are_my_files.txt\r\n|           |-- bin\r\n|           |   |-- dupRadar.r\r\n|           |   |-- edgeR_heatmap_MDS.r\r\n|           |   |-- gtf2bed\r\n|           |   |-- markdown_to_html.r\r\n|           |   |-- merge_featurecounts.py\r\n|           |   |-- RNA-pipeline-from-BAM.sh\r\n|           |   `-- scrape_software_versions.py\r\n|           |-- CHANGELOG.md\r\n|           |-- conf\r\n|           |   |-- aws.config\r\n|           |   |-- base.config\r\n|           |   |-- binac.config\r\n|           |   |-- ccga.config\r\n|           |   |-- cfc.config\r\n|           |   |-- docker.config\r\n|           |   |-- hebbe.config\r\n|           |   |-- igenomes.config\r\n|           |   |-- multiqc_config.yaml\r\n|           |   |-- singularity.config\r\n|           |   |-- uct_hex.config\r\n|           |   |-- uppmax.config\r\n|           |   |-- uppmax-devel.config\r\n|           |   `-- uppmax-modules.config\r\n|           |-- Dockerfile\r\n|           |-- docs\r\n|           |   |-- configuration\r\n|           |   |   |-- adding_your_own.md\r\n|           |   |   |-- aws.md\r\n|           |   |   |-- c3se.md\r\n|           |   |   |-- ccga.md\r\n|           |   |   |-- local.md\r\n|           |   |   |-- qbic.md\r\n|           |   |   |-- reference_genomes.md\r\n|           |   |   `-- uppmax.md\r\n|           |   |-- images\r\n|           |   |   |-- cutadapt_plot.png\r\n|           |   |   |-- dupRadar_plot.png\r\n|           |   |   |-- featureCounts_assignment_plot.png\r\n|           |   |   |-- featureCounts_biotype_plot.png\r\n|           |   |   |-- heatmap.png\r\n|           |   |   |-- IKMB_logo.png\r\n|           |   |   |-- infer_experiment.png\r\n|           |   |   |-- inner_distance_concept.png\r\n|           |   |   |-- junction_saturation.png\r\n|           |   |   |-- mqc_hcplot_hocmzpdjsq.png\r\n|           |   |   |-- mqc_hcplot_ltqchiyxfz.png\r\n|           |   |   |-- mqc_hcplot_wtnqrdhkuc.png\r\n|           |   |   |-- nfcore-rnaseq_logo.ai\r\n|           |   |   |-- nfcore-rnaseq_logo.png\r\n|           |   |   |-- NGI_logo.png\r\n|           |   |   |-- preseq_complexity_curve.png\r\n|           |   |   |-- preseq_plot.png\r\n|           |   |   |-- QBiC_logo.png\r\n|           |   |   |-- read_duplication.png\r\n|           |   |   |-- rseqc_gene_body_coverage_plot.png\r\n|           |   |   |-- rseqc_infer_experiment_plot.png\r\n|           |   |   |-- rseqc_inner_distance_plot.png\r\n|           |   |   |-- rseqc_junction_annotation_junctions_plot.png\r\n|           |   |   |-- rseqc_junction_saturation_plot.png\r\n|           |   |   |-- rseqc_read_distribution_plot.png\r\n|           |   |   |-- rseqc_read_dups_plot.png\r\n|           |   |   |-- saturation.png\r\n|           |   |   |-- SciLifeLab_logo.png\r\n|           |   |   `-- star_alignment_plot.png\r\n|           |   |-- installation.md\r\n|           |   |-- output.md\r\n|           |   |-- README.md\r\n|           |   |-- troubleshooting.md\r\n|           |   `-- usage.md\r\n|           |-- environment.yml\r\n|           |-- LICENSE.md\r\n|           |-- main.nf\r\n|           |-- nextflow.config\r\n|           |-- README.md\r\n|           `-- tests\r\n|               |-- install.sh\r\n|               |-- run_test.sh\r\n|               `-- test_data\r\n|                   `-- ngi-rna_test_set.tar.bz2\r\n`-- ubuntu\r\n    |-- nextflow.config\r\n    |-- results\r\n    |   `-- pipeline_info\r\n    |       `-- nfcore-rnaseq_trace.txt\r\n    `-- work\r\n\r\n16 directories, 91 files\r\n```\r\n\r\nSo, NF was using the persistent volume claim, but didn't find the `data` folder. Here is the `.nextflow.log`:\r\n```\r\nMay-02 16:25:26.503 [main] DEBUG nextflow.cli.Launcher - $> /usr/local/bin/nextflow run nf-core/rnaseq -name nice-jang --genome GRCh37\r\nMay-02 16:25:26.580 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 0.28.0\r\nMay-02 16:25:26.673 [main] INFO  nextflow.cli.CmdRun - Pulling nf-core/rnaseq ...\r\nMay-02 16:25:26.688 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials -:-] -> https://api.github.com/repos/nf-core/rnaseq/contents/nextflow.config\r\nMay-02 16:25:27.505 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials -:-] -> https://api.github.com/repos/nf-core/rnaseq/contents/main.nf\r\nMay-02 16:25:27.799 [main] DEBUG nextflow.scm.RepositoryProvider - Request [credentials -:-] -> https://api.github.com/repos/nf-core/rnaseq\r\nMay-02 16:25:27.946 [main] DEBUG nextflow.scm.AssetManager - Pulling nf-core/rnaseq -- Using remote clone url: https://github.com/nf-core/rnaseq.git\r\nMay-02 16:25:36.047 [main] INFO  nextflow.cli.CmdRun -  downloaded from https://github.com/nf-core/rnaseq.git\r\nMay-02 16:25:36.252 [main] DEBUG nextflow.scm.AssetManager - Git config: /mnt/gluster/projects/nf-core/rnaseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/rnaseq.git\r\nMay-02 16:25:36.253 [main] INFO  nextflow.cli.CmdRun - Launching `nf-core/rnaseq` [nice-jang] - revision: 98ffbce439 [master]\r\nMay-02 16:25:36.263 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /mnt/gluster/projects/nf-core/rnaseq/nextflow.config\r\nMay-02 16:25:36.264 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: nextflow.config\r\nMay-02 16:25:36.265 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /mnt/gluster/projects/nf-core/rnaseq/nextflow.config\r\nMay-02 16:25:36.266 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /mnt/gluster/ubuntu/nextflow.config\r\nMay-02 16:25:36.273 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`\r\nMay-02 16:25:36.529 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`\r\nMay-02 16:25:36.577 [main] DEBUG nextflow.Session - Session uuid: a72f4bb2-f8ad-43b0-a93c-d2e66f391571\r\nMay-02 16:25:36.577 [main] DEBUG nextflow.Session - Run name: nice-jang\r\nMay-02 16:25:36.578 [main] DEBUG nextflow.Session - Executor pool size: 8\r\nMay-02 16:25:36.586 [main] DEBUG nextflow.cli.CmdRun -\r\n  Version: 0.28.0 build 4779\r\n  Modified: 10-03-2018 12:13 UTC (12:13 GMT)\r\n  System: Linux 4.4.0-116-generic\r\n  Runtime: Groovy 2.4.13 on OpenJDK 64-Bit Server VM 1.8.0_151-b12\r\n  Encoding: UTF-8 (UTF-8)\r\n  Process: 9@nice-jang [10.233.64.46]\r\n  CPUs: 8 - Mem: 69.9 GB (62.2 GB) - Swap: 0 (0)\r\nMay-02 16:25:36.609 [main] DEBUG nextflow.Session - Work-dir: /mnt/gluster/ubuntu/work [fuseblk]\r\nMay-02 16:25:36.779 [main] DEBUG nextflow.Session - Session start invoked\r\nMay-02 16:25:36.782 [main] DEBUG nextflow.processor.TaskDispatcher - Dispatcher > start\r\nMay-02 16:25:36.783 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /mnt/gluster/ubuntu/results/pipeline_info/nfcore-rnaseq_trace.txt\r\nMay-02 16:25:36.801 [main] DEBUG nextflow.script.ScriptRunner - > Script parsing\r\nMay-02 16:25:37.174 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution\r\nMay-02 16:25:37.204 [main] DEBUG nextflow.file.FileHelper - Creating a file system instance for provider: S3FileSystemProvider\r\nMay-02 16:25:37.212 [main] DEBUG nextflow.file.FileHelper - AWS S3 config details: {}\r\nMay-02 16:25:37.492 [main] DEBUG nextflow.Channel - files for syntax: glob; folder: data/; pattern: *{1,2}.fastq.gz; options: [:]\r\nMay-02 16:25:37.543 [Thread-2] DEBUG nextflow.Channel - No such file: data/ -- Skipping visit\r\nMay-02 16:25:37.546 [main] INFO  nextflow.Nextflow - ===================================\r\nMay-02 16:25:37.547 [main] INFO  nextflow.Nextflow -  nfcore/rnaseq  ~  version 1.5dev\r\nMay-02 16:25:37.548 [main] INFO  nextflow.Nextflow - ===================================\r\nMay-02 16:25:37.549 [Actor Thread 2] ERROR nextflow.Nextflow - Cannot find any reads matching: data/*{1,2}.fastq.gz\r\nNB: Path needs to be enclosed in quotes!\r\nNB: Path requires at least one * wildcard!\r\nIf this is single-end data, please specify --singleEnd on the command line.\r\n```\r\n\r\nThen, as you recommended, I rerun NF providing the full path, but I got a different error this time:\r\n```\r\nubuntu@k8s-master:~/kubespray$ ./nextflow kuberun nf-core/rnaseq --genome GRCh37 --reads '/mnt/gluster/data/*{1,2}.fastq.gz' -v testpvc:/mnt/gluster/\r\nPod started: cheesy-morse\r\nN E X T F L O W  ~  version 0.28.0\r\nLaunching `nf-core/rnaseq` [cheesy-morse] - revision: 98ffbce439 [master]\r\n===================================\r\n nfcore/rnaseq  ~  version 1.5dev\r\n===================================\r\nERROR ~ No such variable: USER\r\n\r\n -- Check script 'main.nf' at line: 239 or see '.nextflow.log' file for more details\r\n```\r\n\r\nThe `.nextflow.log` in this case is:\r\n```\r\nMay-02 16:40:26.033 [main] DEBUG nextflow.cli.Launcher - $> /usr/local/bin/nextflow run nf-core/rnaseq -name cheesy-morse --genome GRCh37 --reads /mnt/gluster/data/*{1,2}.fastq.gz\r\nMay-02 16:40:26.136 [main] INFO  nextflow.cli.CmdRun - N E X T F L O W  ~  version 0.28.0\r\nMay-02 16:40:26.498 [main] DEBUG nextflow.scm.AssetManager - Git config: /mnt/gluster/projects/nf-core/rnaseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/rnaseq.git\r\nMay-02 16:40:26.568 [main] DEBUG nextflow.scm.AssetManager - Git config: /mnt/gluster/projects/nf-core/rnaseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/rnaseq.git\r\nMay-02 16:40:26.789 [main] DEBUG nextflow.scm.AssetManager - Git config: /mnt/gluster/projects/nf-core/rnaseq/.git/config; branch: master; remote: origin; url: https://github.com/nf-core/rnaseq.git\r\nMay-02 16:40:26.789 [main] INFO  nextflow.cli.CmdRun - Launching `nf-core/rnaseq` [cheesy-morse] - revision: 98ffbce439 [master]\r\nMay-02 16:40:29.993 [main] DEBUG nextflow.config.ConfigBuilder - Found config base: /mnt/gluster/projects/nf-core/rnaseq/nextflow.config\r\nMay-02 16:40:29.994 [main] DEBUG nextflow.config.ConfigBuilder - Found config local: nextflow.config\r\nMay-02 16:40:29.995 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /mnt/gluster/projects/nf-core/rnaseq/nextflow.config\r\nMay-02 16:40:29.995 [main] DEBUG nextflow.config.ConfigBuilder - Parsing config file: /mnt/gluster/ubuntu/nextflow.config\r\nMay-02 16:40:30.003 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`\r\nMay-02 16:40:30.250 [main] DEBUG nextflow.config.ConfigBuilder - Applying config profile: `standard`\r\nMay-02 16:40:30.298 [main] DEBUG nextflow.Session - Session uuid: f78746f8-da81-43c1-81b5-8280c504b6c5\r\nMay-02 16:40:30.298 [main] DEBUG nextflow.Session - Run name: cheesy-morse\r\nMay-02 16:40:30.299 [main] DEBUG nextflow.Session - Executor pool size: 8\r\nMay-02 16:40:30.307 [main] DEBUG nextflow.cli.CmdRun -\r\n  Version: 0.28.0 build 4779\r\n  Modified: 10-03-2018 12:13 UTC (12:13 GMT)\r\n  System: Linux 4.4.0-116-generic\r\n  Runtime: Groovy 2.4.13 on OpenJDK 64-Bit Server VM 1.8.0_151-b12\r\n  Encoding: UTF-8 (UTF-8)\r\n  Process: 10@cheesy-morse [10.233.64.50]\r\n  CPUs: 8 - Mem: 69.9 GB (62.3 GB) - Swap: 0 (0)\r\nMay-02 16:40:30.328 [main] DEBUG nextflow.Session - Work-dir: /mnt/gluster/ubuntu/work [fuseblk]\r\nMay-02 16:40:30.482 [main] DEBUG nextflow.Session - Session start invoked\r\nMay-02 16:40:30.485 [main] DEBUG nextflow.processor.TaskDispatcher - Dispatcher > start\r\nMay-02 16:40:30.486 [main] DEBUG nextflow.trace.TraceFileObserver - Flow starting -- trace file: /mnt/gluster/ubuntu/results/pipeline_info/nfcore-rnaseq_trace.txt\r\nMay-02 16:40:30.514 [main] DEBUG nextflow.script.ScriptRunner - > Script parsing\r\nMay-02 16:40:30.870 [main] DEBUG nextflow.script.ScriptRunner - > Launching execution\r\nMay-02 16:40:30.900 [main] DEBUG nextflow.file.FileHelper - Creating a file system instance for provider: S3FileSystemProvider\r\nMay-02 16:40:30.907 [main] DEBUG nextflow.file.FileHelper - AWS S3 config details: {}\r\nMay-02 16:40:31.186 [main] DEBUG nextflow.Channel - files for syntax: glob; folder: /mnt/gluster/data/; pattern: *{1,2}.fastq.gz; options: [:]\r\nMay-02 16:40:31.196 [main] INFO  nextflow.Nextflow - ===================================\r\nMay-02 16:40:31.196 [main] INFO  nextflow.Nextflow -  nfcore/rnaseq  ~  version 1.5dev\r\nMay-02 16:40:31.198 [main] INFO  nextflow.Nextflow - ===================================\r\nMay-02 16:40:31.201 [main] DEBUG nextflow.Session - Session aborted -- Cause: No such property: USER for class: _nf_script_0c6d925f\r\nMay-02 16:40:31.224 [main] DEBUG nextflow.Session - The following nodes are still active:\r\n  [operator] ifEmpty\r\n  [operator] into\r\n\r\nMay-02 16:40:31.230 [main] ERROR nextflow.cli.Launcher - @unknown\r\ngroovy.lang.MissingPropertyException: No such property: USER for class: _nf_script_0c6d925f\r\n        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)\r\n        at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:52)\r\n        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:307)\r\n        at _nf_script_0c6d925f.run(_nf_script_0c6d925f:239)\r\n        at nextflow.script.ScriptRunner.run(ScriptRunner.groovy:341)\r\n        at nextflow.script.ScriptRunner.execute(ScriptRunner.groovy:165)\r\n        at nextflow.cli.CmdRun.run(CmdRun.groovy:223)\r\n        at nextflow.cli.Launcher.run(Launcher.groovy:428)\r\n        at nextflow.cli.Launcher.main(Launcher.groovy:582)\r\n```\r\n\r\nDo you have any idea what the problem can be? Sorry for such a long text...\r\n"},"comment":{"url":"https://api.github.com/repos/nextflow-io/nextflow/issues/comments/386266468","html_url":"https://github.com/nextflow-io/nextflow/issues/676#issuecomment-386266468","issue_url":"https://api.github.com/repos/nextflow-io/nextflow/issues/676","id":386266468,"user":{"login":"pditommaso","id":816968,"avatar_url":"https://avatars0.githubusercontent.com/u/816968?v=4","gravatar_id":"","url":"https://api.github.com/users/pditommaso","html_url":"https://github.com/pditommaso","followers_url":"https://api.github.com/users/pditommaso/followers","following_url":"https://api.github.com/users/pditommaso/following{/other_user}","gists_url":"https://api.github.com/users/pditommaso/gists{/gist_id}","starred_url":"https://api.github.com/users/pditommaso/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pditommaso/subscriptions","organizations_url":"https://api.github.com/users/pditommaso/orgs","repos_url":"https://api.github.com/users/pditommaso/repos","events_url":"https://api.github.com/users/pditommaso/events{/privacy}","received_events_url":"https://api.github.com/users/pditommaso/received_events","type":"User","site_admin":false},"created_at":"2018-05-03T11:37:05Z","updated_at":"2018-05-03T11:37:05Z","author_association":"OWNER","body":"*good :)"}}
{"actor":{"display_login":"pditommaso"}}