{"action":"created","issue":{"url":"https://api.github.com/repos/aws/aws-sdk-go/issues/1993","repository_url":"https://api.github.com/repos/aws/aws-sdk-go","labels_url":"https://api.github.com/repos/aws/aws-sdk-go/issues/1993/labels{/name}","comments_url":"https://api.github.com/repos/aws/aws-sdk-go/issues/1993/comments","events_url":"https://api.github.com/repos/aws/aws-sdk-go/issues/1993/events","html_url":"https://github.com/aws/aws-sdk-go/issues/1993","id":332563481,"node_id":"MDU6SXNzdWUzMzI1NjM0ODE=","number":1993,"title":"[Feature request] Refresh credentials from file","user":{"login":"mikkeloscar","id":128566,"node_id":"MDQ6VXNlcjEyODU2Ng==","avatar_url":"https://avatars0.githubusercontent.com/u/128566?v=4","gravatar_id":"","url":"https://api.github.com/users/mikkeloscar","html_url":"https://github.com/mikkeloscar","followers_url":"https://api.github.com/users/mikkeloscar/followers","following_url":"https://api.github.com/users/mikkeloscar/following{/other_user}","gists_url":"https://api.github.com/users/mikkeloscar/gists{/gist_id}","starred_url":"https://api.github.com/users/mikkeloscar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikkeloscar/subscriptions","organizations_url":"https://api.github.com/users/mikkeloscar/orgs","repos_url":"https://api.github.com/users/mikkeloscar/repos","events_url":"https://api.github.com/users/mikkeloscar/events{/privacy}","received_events_url":"https://api.github.com/users/mikkeloscar/received_events","type":"User","site_admin":false},"labels":[{"id":323821829,"node_id":"MDU6TGFiZWwzMjM4MjE4Mjk=","url":"https://api.github.com/repos/aws/aws-sdk-go/labels/Closing%20Soon%20-%20Feedback%20Requested","name":"Closing Soon - Feedback Requested","color":"c7def8","default":false},{"id":172085857,"node_id":"MDU6TGFiZWwxNzIwODU4NTc=","url":"https://api.github.com/repos/aws/aws-sdk-go/labels/Feature%20Request","name":"Feature Request","color":"bfe5bf","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-06-14T21:02:30Z","updated_at":"2018-08-25T14:06:01Z","closed_at":null,"author_association":"NONE","body":"This is a feature request to support reading temporary credentials from a file i.e. the shared credentials file, and refresh those credentials when they expire assuming the file has been updated by some external source.\r\n\r\nOther credential providers like the assume role provider implements refresh of credentials and the [docstring for the SharedCredentialsProvider](https://github.com/aws/aws-sdk-go/blob/68bfb358c2f55d3d9fcba781854d546f57149ddc/aws/credentials/shared_credentials_provider.go#L21-L24) also suggests to support this (at least how I read it) but unfortunately [it only reads the credentials once](https://github.com/aws/aws-sdk-go/blob/68bfb358c2f55d3d9fcba781854d546f57149ddc/aws/credentials/shared_credentials_provider.go#L62-L67) and implements a [\"fake\" IsExpired() method](https://github.com/aws/aws-sdk-go/blob/68bfb358c2f55d3d9fcba781854d546f57149ddc/aws/credentials/shared_credentials_provider.go#L71-L74).\r\n\r\nGiven that you can store temporary credentials in the shared credentials file already it would make sense if the SDK could automatically reload the file once the credentials expire. This way you could have some external tool that refreshes the credentials in the file and the SDK would not have to be restarted to reload them.\r\n\r\nMy specific use case is to solve a problem of distributing AWS IAM credentials to Pods in Kubernetes clusters hopefully helping a lot of EKS users in the future!\r\nI have written a [controller](https://github.com/mikkeloscar/kube-aws-iam-controller) that can do this ([README explains how it solves some problems not fixed in other solutions](https://github.com/mikkeloscar/kube-aws-iam-controller/blob/master/README.md#aws-iam-controller-for-kubernetes)).\r\nHowever it relies on the fact that the AWS SDKs can refresh credentials from a file, which unfortunately isn't the case at the moment and the reason why I'm reaching out.\r\n\r\nI realize that this fix needs to be done in every AWS SDK, but Go is what I'm most familiar with so my hope is I can start the discussion here and maybe you can help me get in contact with the right people in case this needs coordination across different AWS SDK implementations.\r\n\r\nI have already talked with the EKS team about this, but they did not have bandwidth to focus on this topic at the moment.\r\n\r\nI have done my own quick implementation in the Go SDK. Feedback very welcome also if I should approach it differently. E.g. I have not considered any possible backwards compatible issues with this change.\r\n\r\nhttps://github.com/aws/aws-sdk-go/compare/master...mikkeloscar:file-refresh\r\n\r\nFor reference here is a [community driven comparison of solutions to the problem of distributing AWS IAM credentials in Kubernetes clusters](https://docs.google.com/document/d/1rn-v2TNH9k4Oz-VuaueP77ANE5p-5Ua89obK2JaArfg/edit).\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/aws/aws-sdk-go/issues/comments/415971588","html_url":"https://github.com/aws/aws-sdk-go/issues/1993#issuecomment-415971588","issue_url":"https://api.github.com/repos/aws/aws-sdk-go/issues/1993","id":415971588,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNTk3MTU4OA==","user":{"login":"mikkeloscar","id":128566,"node_id":"MDQ6VXNlcjEyODU2Ng==","avatar_url":"https://avatars0.githubusercontent.com/u/128566?v=4","gravatar_id":"","url":"https://api.github.com/users/mikkeloscar","html_url":"https://github.com/mikkeloscar","followers_url":"https://api.github.com/users/mikkeloscar/followers","following_url":"https://api.github.com/users/mikkeloscar/following{/other_user}","gists_url":"https://api.github.com/users/mikkeloscar/gists{/gist_id}","starred_url":"https://api.github.com/users/mikkeloscar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mikkeloscar/subscriptions","organizations_url":"https://api.github.com/users/mikkeloscar/orgs","repos_url":"https://api.github.com/users/mikkeloscar/repos","events_url":"https://api.github.com/users/mikkeloscar/events{/privacy}","received_events_url":"https://api.github.com/users/mikkeloscar/received_events","type":"User","site_admin":false},"created_at":"2018-08-25T14:06:01Z","updated_at":"2018-08-25T14:06:01Z","author_association":"NONE","body":"@jasdel thanks for providing an update on this!\r\n\r\n> Our biggest concern with this file being refreshed is two factor. Usage outside of Kubernetes, and that the shared configuration and shared credential files are interchangeable for the most part. This means that there are non-credential related information that would be refreshed. The additional information in the files doesn't make sense to be refreshed during the lifetime of an application.\r\n\r\nIf this is such a big concern, then let me propose a new credentials provider for this use case. We could call it `RefreshableFileCredentials` or something like that and it could rely on a file with the same content as what you get from an assume role i.e.:\r\n\r\n```json\r\n{\r\n    \"AccessKeyId\": \"ACCESS_KEY_ID\",\r\n    \"Expiration\": \"EXPIRATION_DATE\",\r\n    \"RoleArn\": \"TASK_ROLE_ARN\",\r\n    \"SecretAccessKey\": \"SECRET_ACCESS_KEY\",\r\n    \"Token\": \"SECURITY_TOKEN_STRING\"\r\n}\r\n```\r\n\r\nIt could even be enabled ONLY when a certain environment variable is set, e.g.: `AWS_REFRESHABLE_FILE_CREDENTIALS=/path/to/file`. This way it would not affect users outside of Kubernetes (or other environments where this could be useful).\r\n\r\nI don't mind writing a proper proposal for how this would work, I just don't want to spend time on it if there's no chance it could be considered.\r\n\r\nBtw. today I looked into the code of the [AWS SDK for Java](https://github.com/aws/aws-sdk-java) and found out that it does support some form of file refreshing. It does so with a fixed refresh interval of [5 min.](https://github.com/aws/aws-sdk-java/blob/9cbfa17dd0dd58efe5f3f6db46a95863820522fd/aws-java-sdk-core/src/main/java/com/amazonaws/auth/profile/ProfileCredentialsProvider.java#L37) and [reloads the file](https://github.com/aws/aws-sdk-java/blob/9cbfa17dd0dd58efe5f3f6db46a95863820522fd/aws-java-sdk-core/src/main/java/com/amazonaws/auth/profile/ProfilesConfigFile.java#L169-L175) if it was changed since last check. While this is not the nicest solution, it actually solves the problem. I made a simple [example application](https://github.com/mikkeloscar/kube-aws-iam-controller-java-example) to verify that it does indeed work. This basically means the problem is solved for all JVM users which is nice.\r\n\r\nI don't suggest that it should be done the same way for the Go SDK, but considering that the Java SDK has support, would you be more willing to add support (in some form) to the Go SDK?\r\n\r\n> A http endpoint for retrieving credentials per pod provides the best flexibility for applications of users.\r\n\r\nI don't agree. Users don't care how it's implemented. They care that it just works and they don't have to jump through a million hoops. To make an HTTP endpoint implementation \"just work\" is much more complex than what I propose. For reference this is the current [HTTP endpoint based proposal from the community](https://github.com/kubernetes/community/pull/2329). It's very complex..\r\n\r\n> I think further discussing `EC2MetadataDisableTimeoutOverride` as an environment variable makes absolute sense. Ideally this environment variable would be usable across all the SDKs.\r\n\r\nI would really prefer to spend effort on fixing the root cause instead of implementing hacks.\r\nFor me this is equivalent to putting sleeps in your code to work around race conditions.\r\nThat's why I don't like it because it moves the problem away from the SDKs to other components that becomes harder to implement in a good way because of that. When a change is needed to all SDKs anyway, why not make a change that can solve the problem at the lowest level, and solves the problem for good?"}}
{"actor":{"display_login":"mikkeloscar"}}