{"action":"created","issue":{"url":"https://api.github.com/repos/aws/aws-cli/issues/1454","repository_url":"https://api.github.com/repos/aws/aws-cli","labels_url":"https://api.github.com/repos/aws/aws-cli/issues/1454/labels{/name}","comments_url":"https://api.github.com/repos/aws/aws-cli/issues/1454/comments","events_url":"https://api.github.com/repos/aws/aws-cli/issues/1454/events","html_url":"https://github.com/aws/aws-cli/issues/1454","id":101519428,"node_id":"MDU6SXNzdWUxMDE1MTk0Mjg=","number":1454,"title":"aws s3 cp doesn't work with prefix","user":{"login":"fabiocesari","id":8038626,"node_id":"MDQ6VXNlcjgwMzg2MjY=","avatar_url":"https://avatars3.githubusercontent.com/u/8038626?v=4","gravatar_id":"","url":"https://api.github.com/users/fabiocesari","html_url":"https://github.com/fabiocesari","followers_url":"https://api.github.com/users/fabiocesari/followers","following_url":"https://api.github.com/users/fabiocesari/following{/other_user}","gists_url":"https://api.github.com/users/fabiocesari/gists{/gist_id}","starred_url":"https://api.github.com/users/fabiocesari/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fabiocesari/subscriptions","organizations_url":"https://api.github.com/users/fabiocesari/orgs","repos_url":"https://api.github.com/users/fabiocesari/repos","events_url":"https://api.github.com/users/fabiocesari/events{/privacy}","received_events_url":"https://api.github.com/users/fabiocesari/received_events","type":"User","site_admin":false},"labels":[{"id":81772671,"node_id":"MDU6TGFiZWw4MTc3MjY3MQ==","url":"https://api.github.com/repos/aws/aws-cli/labels/investigating","name":"investigating","color":"006b75","default":false},{"id":90091147,"node_id":"MDU6TGFiZWw5MDA5MTE0Nw==","url":"https://api.github.com/repos/aws/aws-cli/labels/s3","name":"s3","color":"eb6420","default":false},{"id":289123970,"node_id":"MDU6TGFiZWwyODkxMjM5NzA=","url":"https://api.github.com/repos/aws/aws-cli/labels/s3filters","name":"s3filters","color":"eb6420","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":31,"created_at":"2015-08-17T22:00:08Z","updated_at":"2018-07-09T04:18:02Z","closed_at":null,"author_association":"NONE","body":"From a bucket containing millions of files, I want do download a few thousand based on their prefix. \n\nNote: using --exclude \"*\" --include \"myprefix\" works, but is impractical since it lists all the files and applies the filter afterwards.\n\nThe problem is aws s3 cp adds a trailing \"/\" after the prefix I specify. In the example below you can see that logs/2015-08-17 becomes logs/2015-08-17/ \nThis additional \"/\" leads to an empty result set, since the keys in the bucket have the form logs/YYYY-MM-DD-HH-MM-SS-RANDOM).\n\n> aws s3 cp s3://mybucket/logs/2015-08-17 . --recursive --dryrun --debug\n\n2015-08-17 21:48:32,026 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.7.27 Python/2.7.6 Linux/3.14.13-c9, botocore version: 0.108.0\n2015-08-17 21:48:32,026 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', 'cp', 's3://mybucket/logs/2015-08-17', '.', '--recursive', '--dryrun', '--debug']\n....\n2015-08-17 21:48:32,088 - MainThread - botocore.endpoint - DEBUG - Making request for <botocore.model.OperationModel object at 0x7ff9d3ec3e50> (verify_ssl=True) with params: {'query_string': {u'prefix': u'logs/2015-08-17/', u'encoding-type': 'url'}, 'headers': {}, 'url_path': u'/mybucket', 'body': '', 'method': u'GET'}\n"},"comment":{"url":"https://api.github.com/repos/aws/aws-cli/issues/comments/403355248","html_url":"https://github.com/aws/aws-cli/issues/1454#issuecomment-403355248","issue_url":"https://api.github.com/repos/aws/aws-cli/issues/1454","id":403355248,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMzM1NTI0OA==","user":{"login":"clandolfi","id":40974085,"node_id":"MDQ6VXNlcjQwOTc0MDg1","avatar_url":"https://avatars3.githubusercontent.com/u/40974085?v=4","gravatar_id":"","url":"https://api.github.com/users/clandolfi","html_url":"https://github.com/clandolfi","followers_url":"https://api.github.com/users/clandolfi/followers","following_url":"https://api.github.com/users/clandolfi/following{/other_user}","gists_url":"https://api.github.com/users/clandolfi/gists{/gist_id}","starred_url":"https://api.github.com/users/clandolfi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clandolfi/subscriptions","organizations_url":"https://api.github.com/users/clandolfi/orgs","repos_url":"https://api.github.com/users/clandolfi/repos","events_url":"https://api.github.com/users/clandolfi/events{/privacy}","received_events_url":"https://api.github.com/users/clandolfi/received_events","type":"User","site_admin":false},"created_at":"2018-07-09T04:18:02Z","updated_at":"2018-07-09T04:18:02Z","author_association":"NONE","body":"+1\r\nas andychase mentions above, an update would be great now that the footwork was done...\r\n\r\n1,350,000 objects must be listed prior to returning the 20 we need? This basically renders s3 useless for our use case...\r\n\r\n"}}
{"actor":{"display_login":"clandolfi"}}