{"action":"created","issue":{"url":"https://api.github.com/repos/Azure/azure-storage-azcopy/issues/87","repository_url":"https://api.github.com/repos/Azure/azure-storage-azcopy","labels_url":"https://api.github.com/repos/Azure/azure-storage-azcopy/issues/87/labels{/name}","comments_url":"https://api.github.com/repos/Azure/azure-storage-azcopy/issues/87/comments","events_url":"https://api.github.com/repos/Azure/azure-storage-azcopy/issues/87/events","html_url":"https://github.com/Azure/azure-storage-azcopy/issues/87","id":369811740,"node_id":"MDU6SXNzdWUzNjk4MTE3NDA=","number":87,"title":"Transfer of .pst file to Office 365 cannot complete with AzCopy for mac","user":{"login":"epgui","id":1992822,"node_id":"MDQ6VXNlcjE5OTI4MjI=","avatar_url":"https://avatars3.githubusercontent.com/u/1992822?v=4","gravatar_id":"","url":"https://api.github.com/users/epgui","html_url":"https://github.com/epgui","followers_url":"https://api.github.com/users/epgui/followers","following_url":"https://api.github.com/users/epgui/following{/other_user}","gists_url":"https://api.github.com/users/epgui/gists{/gist_id}","starred_url":"https://api.github.com/users/epgui/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/epgui/subscriptions","organizations_url":"https://api.github.com/users/epgui/orgs","repos_url":"https://api.github.com/users/epgui/repos","events_url":"https://api.github.com/users/epgui/events{/privacy}","received_events_url":"https://api.github.com/users/epgui/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-10-13T15:20:07Z","updated_at":"2018-10-15T15:28:04Z","closed_at":null,"author_association":"NONE","body":"### Which version of the AzCopy was used? \r\n##### Note: The version is visible when running AzCopy without any argument\r\n\r\n- AzCopy 10.0.2-Preview\r\n\r\n### Which platform are you using? (ex: Windows, Mac, Linux)\r\n\r\n- macOS High Sierra, 10.13.6 (17G65)\r\n\r\n### What command did you run?\r\n##### Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.\r\n\r\n- The following command was run, with URL parameters and file paths replaced by underscores for privacy and security reasons:\r\n\r\n```\r\n./azcopy cp \"___/backup.pst\" \"https://___.blob.core.windows.net/ingestiondata?sv=___&sr=___&si=___&sig=___&se=___\"\r\n```\r\n\r\nNote that the [instructions for Windows](https://docs.microsoft.com/en-us/office365/securitycompliance/use-network-upload-to-import-pst-files) specify the following command:\r\n\r\n```\r\nAzCopy.exe /Source:<Location of PST files> /Dest:<SAS URL> /V:<Log file location> /Y\r\n```\r\n\r\nwith the following comment regarding `/Y`:\r\n\r\n> This required switch allows the use of write-only SAS tokens when you upload the PST files to the Azure storage location. The SAS URL you obtained in step 1 (and specified in /Dest: parameter) is a write-only SAS URL, which is why you must include this switch. Note that a write-only SAS URL will not prevent you from using the Azure Storage Explorer to view a list of the PST files uploaded to the Azure storage location.\r\n\r\nI am unsure how to add the equivalent tag here, as I have found no pointers in the documentation or help pages.\r\n\r\n### What problem was encountered?\r\n\r\n- AzCopy ran silently for a very long time and ended with a failed transfer:\r\n\r\n```\r\nJob a0eef242-719d-c04f-4cd7-8765dcf1b232 has started\r\n\r\na0eef242-719d-c04f-4cd7-8765dcf1b232.log file created in /___/.azcopy\r\n0 Done, 1 Failed, 0 Pending, 0 Skipped, 1 Total\r\n```\r\n\r\nIn the log file I have the following header:\r\n\r\n```\r\n2018/10/12 03:31:09 AzcopVersion  10.0.2-Preview\r\n2018/10/12 03:31:09 OS-Environment  darwin\r\n2018/10/12 03:31:09 OS-Architecture  amd64\r\n2018/10/12 03:31:09 Job-Command cp ___/backup.pst https://___.blob.core.windows.net/ingestiondata?se=___&si=___&sig=___&sr=___&sv=___\r\n2018/10/12 03:31:09 JobID=dfb0ba05-f63c-5748-62e2-8b5f5aa5745a, credential type: Anonymous\r\n2018/10/12 03:31:09 scheduling JobID=dfb0ba05-f63c-5748-62e2-8b5f5aa5745a, Part#=0, Transfer#=0, priority=0\r\n2018/10/12 03:31:09 INFO: [P#0-T#0] has worker 213 which is processing TRANSFER\r\n2018/10/12 03:31:09 INFO: [P#0-T#0] Starting transfer: Source \"___/backup.pst\" Destination \"https://___.blob.core.windows.net/ingestiondata/backup.pst?se=___&si=___&sig=___&sr=___&sv=___\". Specified chunk size 8388608\r\n```\r\n\r\nFollowed by a BUNCH of these that all have the exact same timestamp and \"try\" number (Try=1):\r\n\r\n```\r\n2018/10/12 03:31:09 ==> OUTGOING REQUEST (Try=1)\r\nPUT https://___.blob.core.windows.net/ingestiondata/backup.pst?blockid=___&comp=___&se=___&si=___&sig=___&sr=___&sv=___&timeout=901\r\nContent-Length: [8388608]\r\nUser-Agent: [AzCopy/v10.0.2-Preview Azure-Storage/0.1 (go1.10.3; darwin)]\r\nX-Ms-Client-Request-Id: [7cb4a66d-c7d6-4b5e-7d91-6557c2855b95]\r\nX-Ms-Version: [2018-03-28]\r\n```\r\n\r\nAnd then a bunch of these (whitespace may be wonky, as I have trouble copying from vim):\r\n\r\n```\r\n2018/10/12 03:35:27 ==> REQUEST/RESPONSE (Try=1/4m17.34582974s[SLOW >3s], OpTime=4m17.345889972s) -- REQUEST ERROR\r\n    PUT https://___.blob.core.windows.net/ingestiondata/backup.pst?blockid=___&comp=___&se=___&si=___&sig=___&sr=___&sv=___&timeout=901\r\n    Content-Length: [8388608]\r\n    User-Agent: [AzCopy/v10.0.2-Preview Azure-Storage/0.1 (go1.10.3; darwin)]\r\n    X-Ms-Client-Request-Id: [1db006dc-c187-4242-6a08-d290227a7d60]\r\n    X-Ms-Version: [2018-03-28]\r\n    --------------------------------------------------------------------------------\r\n    ERROR:\r\n-> github.com/Azure/azure-storage-azcopy/ste.newAzcopyHTTPClientFactory.func1.1, /go/src/github.com/Azure/azure-storage-azcopy/ste/mgr-JobPartMgr.go:95\r\n HTTP request failed\r\n \r\nPut https://___.blob.core.windows.net/ingestiondata/backup.pst?blockid=___&comp=___&se=___&si=___&sig=___&sr=___&sv=___&timeout=901: read tcp 10.254.246.200:52028->52.239.148.74:443: read: connection reset by peer\r\n\r\ngoroutine 264 [running]:\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob.stack(0xc42014e310, 0xc420a52100, 0x0)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob/zc_policy_request_log.go:146 +0xa7\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob.NewRequestLogPolicyFactory.func1.1(0x461c020, 0xc4209828a0, 0xc420996800, 0x45b502b, 0xc, 0x45      b4175, 0xa)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob/zc_policy_request_log.go:96 +0x665\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc42096c8c0, 0x461c020, 0xc4209828a0, 0xc420996800, 0xa, 0x452fa80, 0xc420044800, 0xc42072f8      d0)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline/core.go:42 +0x44\r\ngithub.com/Azure/azure-storage-azcopy/ste.NewVersionPolicyFactory.func1.1(0x461c020, 0xc4209828a0, 0xc420996800, 0x0, 0xc4209b7118, 0x402bcd4, 0x45d5b90)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/ste/mgr-JobPartMgr.go:59 +0xe8\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc4209053a0, 0x461c020, 0xc4209828a0, 0xc420996800, 0xc4209b7128, 0xc4209b71e8, 0x4119083, 0      xc4209828b0)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline/core.go:42 +0x44\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob.responderPolicy.Do(0x46186a0, 0xc4209053a0, 0xc420744d20, 0x461c020, 0xc4209828a0, 0xc42099680      0, 0xc4209828b0, 0x461bfa0, 0xc42072f8c0, 0x0)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob/zz_generated_responder_policy.go:33 +0x53\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob.anonymousCredentialPolicy.Do(0x46186e0, 0xc4209053c0, 0x461c020, 0xc4209828a0, 0xc420996800, 0      x4913da0, 0x461c020, 0xc4209828a0, 0xc420953660)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob/zc_credential_anonymous.go:54 +0x4f\r\ngithub.com/Azure/azure-storage-azcopy/ste.NewBlobXferRetryPolicyFactory.func1.1(0x461bfa0, 0xc42072f8c0, 0xc420996700, 0x45b9940, 0x16, 0xc4208d39b0, 0x24)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/ste/xfer-retrypolicy.go:362 +0x6c1\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc42096c910, 0x461bfa0, 0xc42072f8c0, 0xc420996700, 0x24, 0x41f7870, 0x454f900, 0xc4208ef6b0      )\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline/core.go:42 +0x44\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob.NewUniqueRequestIDPolicyFactory.func1.1(0x461bfa0, 0xc42072f8c0, 0xc420996700, 0x45b432d, 0xa,       0xc420768180, 0x3b)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob/zc_policy_unique_request_id.go:19 +0x9c\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc4209053e0, 0x461bfa0, 0xc42072f8c0, 0xc420996700, 0x3b, 0xc42000e450, 0xc4208ef710, 0xc420      9b7608)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline/core.go:42 +0x44\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob.NewTelemetryPolicyFactory.func1.1(0x461bfa0, 0xc42072f8c0, 0xc420996700, 0x1, 0x0, 0x1, 0xc420      00e450)\r\n        /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob/zc_policy_telemetry.go:34 +0x9e\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline.PolicyFunc.Do(0xc4208ef710, 0x461bfa0, 0xc42072f8c0, 0xc420996700, 0xc4208ef710, 0x45b24df, 0xc4209b7678, 0      x4013318)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline/core.go:42 +0x44\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline.(*pipeline).Do(0xc42072f880, 0x461bfa0, 0xc42072f8c0, 0x4618700, 0xc420744d20, 0xc420996700, 0x2d, 0xc42079      2035, 0x19, 0x0)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-pipeline-go/pipeline/core.go:128 +0x81\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob.blockBlobClient.StageBlock(0xc420792000, 0x5, 0x0, 0x0, 0x0, 0xc420792008, 0x2d, 0xc420792035,       0x19, 0x0, ...)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob/zz_generated_block_blob.go:262 +0x4a9\r\ngithub.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob.BlockBlobURL.StageBlock(0xc420792000, 0x5, 0x0, 0x0, 0x0, 0xc420792008, 0x2d, 0xc420792035, 0x      19, 0x0, ...)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/vendor/github.com/Azure/azure-storage-blob-go/2018-03-28/azblob/url_block_blob.go:74 +0x131\r\ngithub.com/Azure/azure-storage-azcopy/ste.(*blockBlobUpload).blockBlobUploadFunc.func1(0xdf)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/ste/xfer-localToBlockBlob.go:338 +0x5a1\r\ngithub.com/Azure/azure-storage-azcopy/ste.(*jobsAdmin).transferAndChunkProcessor(0xc4201340c0, 0xdf)\r\n         /go/src/github.com/Azure/azure-storage-azcopy/ste/JobsAdmin.go:216 +0xa6\r\ncreated by github.com/Azure/azure-storage-azcopy/ste.initJobsAdmin\r\n         /go/src/github.com/Azure/azure-storage-azcopy/ste/JobsAdmin.go:160 +0x501\r\n```\r\n\r\n... but yeah, it's a >12MB log file. Is it safe to share all its contents? I can't tell what's sensitive and what's not haha.\r\n\r\n### How can we reproduce the problem in the simplest way?\r\n\r\n- Follow these instructions to migrate old emails backed up into a .pst file into a new Office 365 email server: https://docs.microsoft.com/en-us/office365/securitycompliance/use-network-upload-to-import-pst-files\r\n\r\n### Have you found a mitigation/solution?\r\n\r\n- Not yet... I am trying to find a Windows PC I can borrow to try getting it done with the Windows utility. This is rather frustrating. I really appreciate any insights or advice you may have! Thanks for taking the time to read this.\r\n"},"comment":{"url":"https://api.github.com/repos/Azure/azure-storage-azcopy/issues/comments/429900426","html_url":"https://github.com/Azure/azure-storage-azcopy/issues/87#issuecomment-429900426","issue_url":"https://api.github.com/repos/Azure/azure-storage-azcopy/issues/87","id":429900426,"node_id":"MDEyOklzc3VlQ29tbWVudDQyOTkwMDQyNg==","user":{"login":"seguler","id":22362788,"node_id":"MDQ6VXNlcjIyMzYyNzg4","avatar_url":"https://avatars0.githubusercontent.com/u/22362788?v=4","gravatar_id":"","url":"https://api.github.com/users/seguler","html_url":"https://github.com/seguler","followers_url":"https://api.github.com/users/seguler/followers","following_url":"https://api.github.com/users/seguler/following{/other_user}","gists_url":"https://api.github.com/users/seguler/gists{/gist_id}","starred_url":"https://api.github.com/users/seguler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seguler/subscriptions","organizations_url":"https://api.github.com/users/seguler/orgs","repos_url":"https://api.github.com/users/seguler/repos","events_url":"https://api.github.com/users/seguler/events{/privacy}","received_events_url":"https://api.github.com/users/seguler/received_events","type":"User","site_admin":false},"created_at":"2018-10-15T15:28:04Z","updated_at":"2018-10-15T15:28:04Z","author_association":"MEMBER","body":"Do you mind retrying again after reducing the number of parallel connections ? It seems you got a TCP failure, and I wonder if it is something to do with too many connections.\r\n\r\nSet the environment value on the command line as such (this sets the parallel connections to 8)\r\nexport AZCOPY_CONCURRENCY_VALUE=8\r\n\r\nAnd then in the same session, re-run the AzCopy command."}}
{"actor":{"display_login":"seguler"}}