{"action":"created","issue":{"url":"https://api.github.com/repos/minio/mc/issues/1693","repository_url":"https://api.github.com/repos/minio/mc","labels_url":"https://api.github.com/repos/minio/mc/issues/1693/labels{/name}","comments_url":"https://api.github.com/repos/minio/mc/issues/1693/comments","events_url":"https://api.github.com/repos/minio/mc/issues/1693/events","html_url":"https://github.com/minio/mc/issues/1693","id":150156453,"number":1693,"title":"mirror: Transfer Reliability Features","user":{"login":"shanemadden","id":3057436,"avatar_url":"https://avatars.githubusercontent.com/u/3057436?v=3","gravatar_id":"","url":"https://api.github.com/users/shanemadden","html_url":"https://github.com/shanemadden","followers_url":"https://api.github.com/users/shanemadden/followers","following_url":"https://api.github.com/users/shanemadden/following{/other_user}","gists_url":"https://api.github.com/users/shanemadden/gists{/gist_id}","starred_url":"https://api.github.com/users/shanemadden/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shanemadden/subscriptions","organizations_url":"https://api.github.com/users/shanemadden/orgs","repos_url":"https://api.github.com/users/shanemadden/repos","events_url":"https://api.github.com/users/shanemadden/events{/privacy}","received_events_url":"https://api.github.com/users/shanemadden/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-04-21T18:37:31Z","updated_at":"2016-04-25T17:01:06Z","closed_at":null,"body":"I've been using `mc mirror` to transfer a large amount of small files from an s3 bucket to a local system, and it's working great, though each of the parallel transfers I've been running has been needing a lot of babysitting. Every TCP handshake timeout, TLS handshake timeout, and connection reset exits the `mc` process, necessitating `mc session resume` (by hand or otherwise).\r\n\r\nFor transfers like this, having optional flags allowing for graceful handling of these errors would greatly reduce the amount of human intervention (or wrapper script writing) needed:\r\n\r\n  - `--retries 5` - it'd be great to gracefully try again when a connection hiccup causes an error (but probably not on by default, since it could incur unexpected bandwidth costs?)\r\n  - `--skip-missing` - I'm running into some legacy cruft in the s3 bucket which is causing `The specified key does not exist.` errors (and I'm guessing this might also happen if a file is scanned but then deleted before it can be copied?). Getting past these objects requires removing their line from the session's .data file, then resuming.\r\n\r\nThanks!"},"comment":{"url":"https://api.github.com/repos/minio/mc/issues/comments/214441920","html_url":"https://github.com/minio/mc/issues/1693#issuecomment-214441920","issue_url":"https://api.github.com/repos/minio/mc/issues/1693","id":214441920,"user":{"login":"shanemadden","id":3057436,"avatar_url":"https://avatars.githubusercontent.com/u/3057436?v=3","gravatar_id":"","url":"https://api.github.com/users/shanemadden","html_url":"https://github.com/shanemadden","followers_url":"https://api.github.com/users/shanemadden/followers","following_url":"https://api.github.com/users/shanemadden/following{/other_user}","gists_url":"https://api.github.com/users/shanemadden/gists{/gist_id}","starred_url":"https://api.github.com/users/shanemadden/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shanemadden/subscriptions","organizations_url":"https://api.github.com/users/shanemadden/orgs","repos_url":"https://api.github.com/users/shanemadden/repos","events_url":"https://api.github.com/users/shanemadden/events{/privacy}","received_events_url":"https://api.github.com/users/shanemadden/received_events","type":"User","site_admin":false},"created_at":"2016-04-25T17:01:06Z","updated_at":"2016-04-25T17:01:06Z","body":"> This error should be just reported and the mirror should continue without copying these files already, you want suppress these error messages?\r\n\r\nThe behavior I'm seeing is that the `The specified key does not exist` errors are actually causing `mc` to exit with a status code of 1:\r\n\r\n```\r\n[root@hostname ~]# mc mirror s3/bucket/0/ /mnt/mirror/0/\r\n# runs until encountering a bad key:\r\nmc: <ERROR> Failed to copy ‘https://s3.amazonaws.com/bucket/0/f/3/e’. The specified key does not exist.\r\nmc: <ERROR> Session safely terminated. To resume session ‘mc session resume kOcaYyNu’\r\n\r\n[root@hostname ~]# mc session resume kOcaYyNu\r\n# resume works with the progress bar quickly jumping to where it was, then the key fails again immediately.\r\nmc: <ERROR> Failed to copy ‘https://s3.amazonaws.com/bucket/0/f/3/e’. The specified key does not exist.\r\nmc: <ERROR> Session safely terminated. To resume session ‘mc session resume kOcaYyNu’\r\n\r\n[root@hostname ~]# vi /root/.mc/session/kOcaYyNu.data\r\n# remove the json line belonging to 0/f/3/e by hand\r\n\r\n[root@hostname ~]# mc session resume kOcaYyNu\r\n# proceeds past the missing key\r\n```\r\n\r\n> These errors we will add to be retried in minio-go client.\r\n\r\nGreat, thanks! One more for you that I just encountered for the first time, unsure whether it's common enough to add: `mc: <ERROR> Failed to copy ‘https://s3.amazonaws.com/foo’. Input reader closed pre-maturely. Expected ‘576044’ bytes, but only received ‘543997’ bytes.`\r\n\r\n> This is already retried, do you not see that behavior?\r\n\r\nI only ever saw that error once, so maybe in that one case it was erroring repeatedly (retry working as intended, but the network error was persisting on the retries too)."}}