{"action":"created","issue":{"url":"https://api.github.com/repos/BD2KGenomics/toil/issues/859","repository_url":"https://api.github.com/repos/BD2KGenomics/toil","labels_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/859/labels{/name}","comments_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/859/comments","events_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/859/events","html_url":"https://github.com/BD2KGenomics/toil/issues/859","id":154261904,"number":859,"title":"Unable to import files","user":{"login":"tetron","id":1316612,"avatar_url":"https://avatars.githubusercontent.com/u/1316612?v=3","gravatar_id":"","url":"https://api.github.com/users/tetron","html_url":"https://github.com/tetron","followers_url":"https://api.github.com/users/tetron/followers","following_url":"https://api.github.com/users/tetron/following{/other_user}","gists_url":"https://api.github.com/users/tetron/gists{/gist_id}","starred_url":"https://api.github.com/users/tetron/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tetron/subscriptions","organizations_url":"https://api.github.com/users/tetron/orgs","repos_url":"https://api.github.com/users/tetron/repos","events_url":"https://api.github.com/users/tetron/events{/privacy}","received_events_url":"https://api.github.com/users/tetron/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-05-11T14:45:58Z","updated_at":"2016-05-11T18:50:50Z","closed_at":null,"body":"File import does not give any feedback.  It seems to be adding a file to the S3 bucket but gets stuck.  I hit ^C after about 15 minutes of no progress and no network activity.  It appears to waiting for a response:\r\n\r\n```\r\n$ cwltoil --logDebug --batchSystem=mesos --mesosMaster=mesos-master:5050 --jobStore=aws:us-west-2:cwltoil-test-pa2 main-bwa_freebayes_HG01977.cwl main-bwa_freebayes_HG01977-samples.json \r\n2016-05-11 14:17:31,458 INFO:toil.lib.bioio: Logging set at level: DEBUG\r\n2016-05-11 14:18:04,334 INFO:toil.lib.bioio: Logging set at level: DEBUG\r\n2016-05-11 14:18:04,398 DEBUG:toil.jobStores.aws.jobStore: Instantiating <class 'toil.jobStores.aws.jobStore.AWSJobStore'> for region us-west-2 and name prefix 'cwltoil-test-pa2'\r\n2016-05-11 14:18:08,300 INFO:toil.jobStores.abstractJobStore: The workflow ID is: '75205970-692b-4e0e-b3b7-3f457c55e77b'\r\n2016-05-11 14:18:08,648 DEBUG:toil.jobStores.aws.jobStore: Wrote FileInfo(fileID='a77581f6-ad82-5142-acb7-5a6b01eab3fb', ownerID='891f7db6-e4d9-4221-a58e-ab6cc4395f94', encrypted=False, version='', previousVersion='', content='\\x80\\x02ctoil.co...ryCountq6K\\x00ub.', _numContentChunks=2) bound to <toil.jobStores.aws.jobStore.AWSJobStore object at 0x7f4fb88c3890> for shared file 'config.pickle'.\r\n2016-05-11 14:18:08,734 INFO:toil.jobStores.abstractJobStore: Unable to import 'toil.jobStores.azureJobStore'\r\n\r\n^CTraceback (most recent call last):\r\n  File \"/usr/local/bin/cwltoil\", line 9, in <module>\r\n    load_entry_point('toil==3.2.0a2', 'console_scripts', 'cwltoil')()\r\n  File \"/usr/local/lib/python2.7/dist-packages/toil/cwl/cwltoil.py\", line 580, in main\r\n    adjustFiles(builder.job, functools.partial(writeFile, toil.importFile, {}))\r\n  File \"/usr/local/lib/python2.7/dist-packages/cwltool/process.py\", line 128, in adjustFiles\r\n    adjustFiles(rec[d], op)\r\n  File \"/usr/local/lib/python2.7/dist-packages/cwltool/process.py\", line 131, in adjustFiles\r\n    adjustFiles(d, op)\r\n  File \"/usr/local/lib/python2.7/dist-packages/cwltool/process.py\", line 131, in adjustFiles\r\n    adjustFiles(d, op)\r\n  File \"/usr/local/lib/python2.7/dist-packages/cwltool/process.py\", line 126, in adjustFiles\r\n    rec[\"path\"] = op(rec[\"path\"])\r\n  File \"/usr/local/lib/python2.7/dist-packages/toil/cwl/cwltoil.py\", line 162, in writeFile\r\n    index[x] = (writeFunc(rp), os.path.basename(x))\r\n  File \"/usr/local/lib/python2.7/dist-packages/toil/common.py\", line 653, in importFile\r\n    return self._jobStore.importFile(srcUrl)\r\n  File \"/usr/local/lib/python2.7/dist-packages/toil/jobStores/abstractJobStore.py\", line 259, in importFile\r\n    return self._importFile(findJobStoreForUrl(url), url)\r\n  File \"/usr/local/lib/python2.7/dist-packages/toil/jobStores/aws/jobStore.py\", line 279, in _importFile\r\n    info.copyFrom(srcKey)\r\n  File \"/usr/local/lib/python2.7/dist-packages/toil/jobStores/aws/jobStore.py\", line 846, in copyFrom\r\n    headers=self._s3EncryptionHeaders()).version_id\r\n  File \"/usr/local/lib/python2.7/dist-packages/toil/jobStores/aws/jobStore.py\", line 878, in _copyKey\r\n    headers=headers)\r\n  File \"/usr/local/lib/python2.7/dist-packages/toil/jobStores/aws/jobStore.py\", line 85, in copyKeyMultipart\r\n    headers=headers)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/multipart.py\", line 306, in copy_part_from_key\r\n    query_args=query_args)\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/s3/bucket.py\", line 872, in copy_key\r\n    body = response.read()\r\n  File \"/usr/local/lib/python2.7/dist-packages/boto/connection.py\", line 410, in read\r\n    self._cached_response = http_client.HTTPResponse.read(self)\r\n  File \"/usr/lib/python2.7/httplib.py\", line 588, in read\r\n    return self._read_chunked(amt)\r\n  File \"/usr/lib/python2.7/httplib.py\", line 630, in _read_chunked\r\n    line = self.fp.readline(_MAXLINE + 1)\r\n  File \"/usr/lib/python2.7/socket.py\", line 480, in readline\r\n    data = self._sock.recv(self._rbufsize)\r\n  File \"/usr/lib/python2.7/ssl.py\", line 734, in recv\r\n    return self.read(buflen)\r\n  File \"/usr/lib/python2.7/ssl.py\", line 621, in read\r\n    v = self._sslobj.read(len or 1024)\r\nKeyboardInterrupt\r\n```"},"comment":{"url":"https://api.github.com/repos/BD2KGenomics/toil/issues/comments/218554071","html_url":"https://github.com/BD2KGenomics/toil/issues/859#issuecomment-218554071","issue_url":"https://api.github.com/repos/BD2KGenomics/toil/issues/859","id":218554071,"user":{"login":"hannes-ucsc","id":5143256,"avatar_url":"https://avatars.githubusercontent.com/u/5143256?v=3","gravatar_id":"","url":"https://api.github.com/users/hannes-ucsc","html_url":"https://github.com/hannes-ucsc","followers_url":"https://api.github.com/users/hannes-ucsc/followers","following_url":"https://api.github.com/users/hannes-ucsc/following{/other_user}","gists_url":"https://api.github.com/users/hannes-ucsc/gists{/gist_id}","starred_url":"https://api.github.com/users/hannes-ucsc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hannes-ucsc/subscriptions","organizations_url":"https://api.github.com/users/hannes-ucsc/orgs","repos_url":"https://api.github.com/users/hannes-ucsc/repos","events_url":"https://api.github.com/users/hannes-ucsc/events{/privacy}","received_events_url":"https://api.github.com/users/hannes-ucsc/received_events","type":"User","site_admin":false},"created_at":"2016-05-11T18:50:50Z","updated_at":"2016-05-11T18:50:50Z","body":"Is the destination job store in the same AWS region as the source object's bucket?\r\n\r\nHow big is the file?\r\n\r\n"}}