{"action":"created","issue":{"url":"https://api.github.com/repos/mapbox/rasterio/issues/1706","repository_url":"https://api.github.com/repos/mapbox/rasterio","labels_url":"https://api.github.com/repos/mapbox/rasterio/issues/1706/labels{/name}","comments_url":"https://api.github.com/repos/mapbox/rasterio/issues/1706/comments","events_url":"https://api.github.com/repos/mapbox/rasterio/issues/1706/events","html_url":"https://github.com/mapbox/rasterio/issues/1706","id":454563525,"node_id":"MDU6SXNzdWU0NTQ1NjM1MjU=","number":1706,"title":"GDAL_DISABLE_READDIR_ON_OPEN affects how rasterio handles files on S3","user":{"login":"ungarj","id":3802285,"node_id":"MDQ6VXNlcjM4MDIyODU=","avatar_url":"https://avatars3.githubusercontent.com/u/3802285?v=4","gravatar_id":"","url":"https://api.github.com/users/ungarj","html_url":"https://github.com/ungarj","followers_url":"https://api.github.com/users/ungarj/followers","following_url":"https://api.github.com/users/ungarj/following{/other_user}","gists_url":"https://api.github.com/users/ungarj/gists{/gist_id}","starred_url":"https://api.github.com/users/ungarj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ungarj/subscriptions","organizations_url":"https://api.github.com/users/ungarj/orgs","repos_url":"https://api.github.com/users/ungarj/repos","events_url":"https://api.github.com/users/ungarj/events{/privacy}","received_events_url":"https://api.github.com/users/ungarj/received_events","type":"User","site_admin":false},"labels":[{"id":989674718,"node_id":"MDU6TGFiZWw5ODk2NzQ3MTg=","url":"https://api.github.com/repos/mapbox/rasterio/labels/upstream","name":"upstream","color":"ef5fbf","default":false}],"state":"open","locked":false,"assignee":{"login":"sgillies","id":33697,"node_id":"MDQ6VXNlcjMzNjk3","avatar_url":"https://avatars2.githubusercontent.com/u/33697?v=4","gravatar_id":"","url":"https://api.github.com/users/sgillies","html_url":"https://github.com/sgillies","followers_url":"https://api.github.com/users/sgillies/followers","following_url":"https://api.github.com/users/sgillies/following{/other_user}","gists_url":"https://api.github.com/users/sgillies/gists{/gist_id}","starred_url":"https://api.github.com/users/sgillies/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sgillies/subscriptions","organizations_url":"https://api.github.com/users/sgillies/orgs","repos_url":"https://api.github.com/users/sgillies/repos","events_url":"https://api.github.com/users/sgillies/events{/privacy}","received_events_url":"https://api.github.com/users/sgillies/received_events","type":"User","site_admin":false},"assignees":[{"login":"sgillies","id":33697,"node_id":"MDQ6VXNlcjMzNjk3","avatar_url":"https://avatars2.githubusercontent.com/u/33697?v=4","gravatar_id":"","url":"https://api.github.com/users/sgillies","html_url":"https://github.com/sgillies","followers_url":"https://api.github.com/users/sgillies/followers","following_url":"https://api.github.com/users/sgillies/following{/other_user}","gists_url":"https://api.github.com/users/sgillies/gists{/gist_id}","starred_url":"https://api.github.com/users/sgillies/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sgillies/subscriptions","organizations_url":"https://api.github.com/users/sgillies/orgs","repos_url":"https://api.github.com/users/sgillies/repos","events_url":"https://api.github.com/users/sgillies/events{/privacy}","received_events_url":"https://api.github.com/users/sgillies/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/mapbox/rasterio/milestones/90","html_url":"https://github.com/mapbox/rasterio/milestone/90","labels_url":"https://api.github.com/repos/mapbox/rasterio/milestones/90/labels","id":4646414,"node_id":"MDk6TWlsZXN0b25lNDY0NjQxNA==","number":90,"title":"post-1.1","description":"","creator":{"login":"sgillies","id":33697,"node_id":"MDQ6VXNlcjMzNjk3","avatar_url":"https://avatars2.githubusercontent.com/u/33697?v=4","gravatar_id":"","url":"https://api.github.com/users/sgillies","html_url":"https://github.com/sgillies","followers_url":"https://api.github.com/users/sgillies/followers","following_url":"https://api.github.com/users/sgillies/following{/other_user}","gists_url":"https://api.github.com/users/sgillies/gists{/gist_id}","starred_url":"https://api.github.com/users/sgillies/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sgillies/subscriptions","organizations_url":"https://api.github.com/users/sgillies/orgs","repos_url":"https://api.github.com/users/sgillies/repos","events_url":"https://api.github.com/users/sgillies/events{/privacy}","received_events_url":"https://api.github.com/users/sgillies/received_events","type":"User","site_admin":false},"open_issues":7,"closed_issues":0,"state":"open","created_at":"2019-09-09T22:51:33Z","updated_at":"2019-10-02T14:37:22Z","due_on":null,"closed_at":null},"comments":3,"created_at":"2019-06-11T08:56:30Z","updated_at":"2019-10-03T13:16:39Z","closed_at":null,"author_association":"NONE","body":"rasterio behaves differently towards remote files on S3 depening on whether `GDAL_DISABLE_READDIR_ON_OPEN` is activated or not. This could also affect #987.\r\n\r\nexample 1: access a non-existing file on a bucket requiring access token\r\n------------------------------------------------------------------------\r\n\r\nexpected error message without GDAL setting:\r\n```\r\n$ rio info s3://sentinel-s2-l1c/non_existing.tif\r\n<...>\r\nrasterio.errors.RasterioIOError: Access Denied\r\n```\r\n\r\nunexpected error message using GDAL setting:\r\n```\r\n$ export GDAL_DISABLE_READDIR_ON_OPEN=YES\r\n$ rio info s3://sentinel-s2-l1c/non_existing.tif\r\n<...>\r\nrasterio.errors.RasterioIOError: '/vsis3/sentinel-s2-l1c/non_existing.tif' not recognized as a supported file format.\r\n```\r\n\r\n\r\nexample 2: access a non-existing file on a public bucket requiring no access token\r\n----------------------------------------------------------------------------------\r\n\r\nexpected error message without GDAL setting:\r\n```\r\n$ rio info s3://test-gtiff/non_existing.tif\r\n<...>\r\nrasterio.errors.RasterioIOError: '/vsis3/test-gtiff/non_existing.tif' does not exist in the file system, and is not recognized as a supported dataset name.\r\n```\r\n\r\nunexpected error message using GDAL setting:\r\n```\r\n$ export GDAL_DISABLE_READDIR_ON_OPEN=YES\r\n$ rio info s3://test-gtiff/non_existing.tif\r\n<...>\r\nrasterio.errors.RasterioIOError: '/vsis3/test-gtiff/non_existing.tif' not recognized as a supported file format.\r\n```\r\n\r\nI don't know whether this behavior is intentional but I would rather expect getting a consistent error message independent from `GDAL_DISABLE_READDIR_ON_OPEN`.\r\n\r\nVery cool however would be if a generic `FileNotFoundError` or `RasterioFileNotFoundError` is raised so I can avoid hacks like [this](https://github.com/ungarj/mapchete/blob/1765349934d50f50c4ff6529bf40042a6c471b4c/mapchete/io/raster.py#L335) :)"},"comment":{"url":"https://api.github.com/repos/mapbox/rasterio/issues/comments/537941034","html_url":"https://github.com/mapbox/rasterio/issues/1706#issuecomment-537941034","issue_url":"https://api.github.com/repos/mapbox/rasterio/issues/1706","id":537941034,"node_id":"MDEyOklzc3VlQ29tbWVudDUzNzk0MTAzNA==","user":{"login":"ungarj","id":3802285,"node_id":"MDQ6VXNlcjM4MDIyODU=","avatar_url":"https://avatars3.githubusercontent.com/u/3802285?v=4","gravatar_id":"","url":"https://api.github.com/users/ungarj","html_url":"https://github.com/ungarj","followers_url":"https://api.github.com/users/ungarj/followers","following_url":"https://api.github.com/users/ungarj/following{/other_user}","gists_url":"https://api.github.com/users/ungarj/gists{/gist_id}","starred_url":"https://api.github.com/users/ungarj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ungarj/subscriptions","organizations_url":"https://api.github.com/users/ungarj/orgs","repos_url":"https://api.github.com/users/ungarj/repos","events_url":"https://api.github.com/users/ungarj/events{/privacy}","received_events_url":"https://api.github.com/users/ungarj/received_events","type":"User","site_admin":false},"created_at":"2019-10-03T13:16:39Z","updated_at":"2019-10-03T13:16:39Z","author_association":"NONE","body":"Very cool, thanks @sgillies!"}}
{"actor":{"display_login":"ungarj"}}