{"action":"created","issue":{"url":"https://api.github.com/repos/google/oauth2client/issues/504","repository_url":"https://api.github.com/repos/google/oauth2client","labels_url":"https://api.github.com/repos/google/oauth2client/issues/504/labels{/name}","comments_url":"https://api.github.com/repos/google/oauth2client/issues/504/comments","events_url":"https://api.github.com/repos/google/oauth2client/issues/504/events","html_url":"https://github.com/google/oauth2client/pull/504","id":150467511,"number":504,"title":"Add multiprocess file storage.","user":{"login":"jonparrott","id":250995,"avatar_url":"https://avatars.githubusercontent.com/u/250995?v=3","gravatar_id":"","url":"https://api.github.com/users/jonparrott","html_url":"https://github.com/jonparrott","followers_url":"https://api.github.com/users/jonparrott/followers","following_url":"https://api.github.com/users/jonparrott/following{/other_user}","gists_url":"https://api.github.com/users/jonparrott/gists{/gist_id}","starred_url":"https://api.github.com/users/jonparrott/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jonparrott/subscriptions","organizations_url":"https://api.github.com/users/jonparrott/orgs","repos_url":"https://api.github.com/users/jonparrott/repos","events_url":"https://api.github.com/users/jonparrott/events{/privacy}","received_events_url":"https://api.github.com/users/jonparrott/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/google/oauth2client/labels/cla:%20yes","name":"cla: yes","color":"009800"},{"url":"https://api.github.com/repos/google/oauth2client/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":{"login":"nathanielmanistaatgoogle","id":10131044,"avatar_url":"https://avatars.githubusercontent.com/u/10131044?v=3","gravatar_id":"","url":"https://api.github.com/users/nathanielmanistaatgoogle","html_url":"https://github.com/nathanielmanistaatgoogle","followers_url":"https://api.github.com/users/nathanielmanistaatgoogle/followers","following_url":"https://api.github.com/users/nathanielmanistaatgoogle/following{/other_user}","gists_url":"https://api.github.com/users/nathanielmanistaatgoogle/gists{/gist_id}","starred_url":"https://api.github.com/users/nathanielmanistaatgoogle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nathanielmanistaatgoogle/subscriptions","organizations_url":"https://api.github.com/users/nathanielmanistaatgoogle/orgs","repos_url":"https://api.github.com/users/nathanielmanistaatgoogle/repos","events_url":"https://api.github.com/users/nathanielmanistaatgoogle/events{/privacy}","received_events_url":"https://api.github.com/users/nathanielmanistaatgoogle/received_events","type":"User","site_admin":false},"assignees":[{"login":"nathanielmanistaatgoogle","id":10131044,"avatar_url":"https://avatars.githubusercontent.com/u/10131044?v=3","gravatar_id":"","url":"https://api.github.com/users/nathanielmanistaatgoogle","html_url":"https://github.com/nathanielmanistaatgoogle","followers_url":"https://api.github.com/users/nathanielmanistaatgoogle/followers","following_url":"https://api.github.com/users/nathanielmanistaatgoogle/following{/other_user}","gists_url":"https://api.github.com/users/nathanielmanistaatgoogle/gists{/gist_id}","starred_url":"https://api.github.com/users/nathanielmanistaatgoogle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nathanielmanistaatgoogle/subscriptions","organizations_url":"https://api.github.com/users/nathanielmanistaatgoogle/orgs","repos_url":"https://api.github.com/users/nathanielmanistaatgoogle/repos","events_url":"https://api.github.com/users/nathanielmanistaatgoogle/events{/privacy}","received_events_url":"https://api.github.com/users/nathanielmanistaatgoogle/received_events","type":"User","site_admin":false}],"milestone":null,"comments":19,"created_at":"2016-04-22T20:45:49Z","updated_at":"2016-07-12T19:51:51Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/google/oauth2client/pulls/504","html_url":"https://github.com/google/oauth2client/pull/504","diff_url":"https://github.com/google/oauth2client/pull/504.diff","patch_url":"https://github.com/google/oauth2client/pull/504.patch"},"body":"Towards #470\r\n\r\nThis is not ready for merge, this is an initial sketch of what this would look like. Please note:\r\n\r\n1. Instead of outright replacing `multistore_file`, I am introducing a new module to cover the same functionality named `multiprocess_file_storage`. We can do a release to deprecate `multistore_file` in favor of `multiprocess_file_storage` and then another release to remove it.\r\n2. The interface has changed. Instead of a bunch of `get_storage_{}` functions, there is now just `MultiprocesFileStorage(filename, key)`. The key argument can be used to emulate how the previous helper methods worked, eg. `key = '{}-{}-{}'.format(client_id, user_agent, scope)`.\r\n3. Tests are not done, want to sanity check this first.\r\n\r\n@thobrla, while writing this, I came across some interesting behavioral cases:\r\n1. The storage only holds the lock as long as needed to read/write the credentials file during `put` or `get`, but if at *any* point in time it fails to acquire the lock it will switch to read only mode and will never try locking again.\r\n2. The storage only reads the credentials file once, but writes to it constantly. This means if multiple processes access the store and write, they are probably erasing the other's stuff despite the locking.\r\n\r\nIt seems that `multistore_file` never lived up to its promises, and neither will this class. As I mentioned before, a sqlite-based storage would be a true solution but it would require the cloud sdk and gsutil to take on a third-party **binary** dependency, which may be untenable for you. What are your thoughts?"},"comment":{"url":"https://api.github.com/repos/google/oauth2client/issues/comments/232159375","html_url":"https://github.com/google/oauth2client/pull/504#issuecomment-232159375","issue_url":"https://api.github.com/repos/google/oauth2client/issues/504","id":232159375,"user":{"login":"nathanielmanistaatgoogle","id":10131044,"avatar_url":"https://avatars.githubusercontent.com/u/10131044?v=3","gravatar_id":"","url":"https://api.github.com/users/nathanielmanistaatgoogle","html_url":"https://github.com/nathanielmanistaatgoogle","followers_url":"https://api.github.com/users/nathanielmanistaatgoogle/followers","following_url":"https://api.github.com/users/nathanielmanistaatgoogle/following{/other_user}","gists_url":"https://api.github.com/users/nathanielmanistaatgoogle/gists{/gist_id}","starred_url":"https://api.github.com/users/nathanielmanistaatgoogle/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nathanielmanistaatgoogle/subscriptions","organizations_url":"https://api.github.com/users/nathanielmanistaatgoogle/orgs","repos_url":"https://api.github.com/users/nathanielmanistaatgoogle/repos","events_url":"https://api.github.com/users/nathanielmanistaatgoogle/events{/privacy}","received_events_url":"https://api.github.com/users/nathanielmanistaatgoogle/received_events","type":"User","site_admin":false},"created_at":"2016-07-12T19:51:51Z","updated_at":"2016-07-12T19:51:51Z","body":"I dispute that \"[g]etting an exception of any kind when accessing storage effectively amounts to a cache miss\" - an exception could mean _anything_, from interpreter shutdown to local filesystem corruption to a programming defect in the called system to a programming defect in our own system to global thermonuclear conflict. An exception that comes from `fasteners`, and is not on the documented list of this-is-what-`fasteners`-raises-and-the-circumstances-in-which-`fasteners`-raises-it, is necessarily of process-wide interest. If our code were application code fully in command of everything that happens in the process, I'd be open to a catch-all or catch-most. But we're just a library used by the application. We should only catch the exceptions that the other library (`fasteners`) that we are using is documented to raise and we should only behave when catching those exceptions in ways commensurate with our use of the other library. Declining to bother the application we serve about what happened may seem like a safe assumption to make, but I don't think it's an assumption we should be making at all, regardless of its safety.\r\n\r\nNow that I look at [`acquire` and `release`](http://fasteners.readthedocs.io/en/latest/api/process_lock.html), they're not documented to raise anything. Do we have any operational experience that they raise anything? If not, let's just move forward without any catching until we gain some experience that the library we're using behaves in some way other than as advertised?"}}
{"actor":{"display_login":"nathanielmanistaatgoogle"}}