{"action":"created","issue":{"url":"https://api.github.com/repos/cloudant/python-cloudant/issues/380","repository_url":"https://api.github.com/repos/cloudant/python-cloudant","labels_url":"https://api.github.com/repos/cloudant/python-cloudant/issues/380/labels{/name}","comments_url":"https://api.github.com/repos/cloudant/python-cloudant/issues/380/comments","events_url":"https://api.github.com/repos/cloudant/python-cloudant/issues/380/events","html_url":"https://github.com/cloudant/python-cloudant/issues/380","id":320101769,"number":380,"title":"Reads returning old document versions when reusing client","user":{"login":"h3nry0","id":29688941,"avatar_url":"https://avatars1.githubusercontent.com/u/29688941?v=4","gravatar_id":"","url":"https://api.github.com/users/h3nry0","html_url":"https://github.com/h3nry0","followers_url":"https://api.github.com/users/h3nry0/followers","following_url":"https://api.github.com/users/h3nry0/following{/other_user}","gists_url":"https://api.github.com/users/h3nry0/gists{/gist_id}","starred_url":"https://api.github.com/users/h3nry0/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/h3nry0/subscriptions","organizations_url":"https://api.github.com/users/h3nry0/orgs","repos_url":"https://api.github.com/users/h3nry0/repos","events_url":"https://api.github.com/users/h3nry0/events{/privacy}","received_events_url":"https://api.github.com/users/h3nry0/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-05-03T22:29:10Z","updated_at":"2018-05-14T12:06:14Z","closed_at":null,"author_association":"NONE","body":"I have a serverless function calling a Cloudant database via the [python-cloudant][1] library, with the connection to the client being made outside the function handler so that it is cached, like so:\r\n\r\n    httpAdapter = HTTPAdapter(pool_connections=5, pool_maxsize=100)\r\n    client = Cloudant(cloudantUsername, cloudantPassword, url=cloudantURL, adapter=httpAdapter, connect=True)\r\n\r\n    def update(event, context):\r\n        db = client['db_name']\r\n        document_id = event['pathParameters']['id']\r\n        document = db[document_id]\r\n        document['foo'] = 'bar'\r\n        document.save()\r\n        return json.dumps(document)\r\n\r\n    def get(event, context):\r\n        db = client['db_name']\r\n        document_id = event['pathParameters']['id']\r\n        document = db[document_id]\r\n        return json.dumps(document)\r\n\r\nWhen I do this, I get a very low latency (<100ms) on repeat executions of any kind (create, update, get), because the \"connection\" is reused, which is the intention. (\"connection\" in inverted commas since I know cloudant does not have a concept of keeping an open pipe).\r\n\r\nIf I call `/update` and increment the document `_rev` to `2-XXXX` then call `/get`, the function returns document with `_rev` `2-XXXX` - expected behaviour. \r\n\r\nIf I call `/update` again, `_rev` is correctly incremented to `3-XXXX`, but when I call `/get` again, I get document revision `2-XXXX` (i.e. not the updated version). When this is happening, the update function is executing in ~90ms, while the get is executing in ~10ms (too fast to actually be calling the database).\r\n\r\nI can eliminate this behaviour by either moving the setup of the Cloudant client inside the function handler like so:\r\n\r\n    httpAdapter = HTTPAdapter(pool_connections=5, pool_maxsize=100)\r\n\r\n    def update(event, context):\r\n        client = Cloudant(cloudantUsername, cloudantPassword, url=cloudantURL, adapter=httpAdapter, connect=True)\r\n        db = client['db_name']\r\n        document_id = event['pathParameters']['id']\r\n        document = db[document_id]\r\n        document['foo'] = 'bar'\r\n        document.save()\r\n        return json.dumps(document)\r\n\r\n    def get(event, context):\r\n        client = Cloudant(cloudantUsername, cloudantPassword, url=cloudantURL, adapter=httpAdapter, connect=True)\r\n        db = client['db_name']\r\n        document_id = event['pathParameters']['id']\r\n        document = db[document_id]\r\n        return json.dumps(document)\r\n\r\nor I can explicitly call `document.fetch()` in the get handler:\r\n\r\n    def get(event, context):\r\n        db = client['db_name']\r\n        document_id = event['pathParameters']['id']\r\n        document = db[document_id]\r\n        document.fetch()\r\n        return json.dumps(document)\r\n\r\nIn either case, `/get` always returns the correct revision of the document.\r\n\r\nGetting an old version of the document is unexpected since I believed putting the client setup outside the handler would only cache the client between executions, but calls to the database would still be made each time the handler is called. It looks like they are being made when `/update` is called, presumably since `document.save()` forces it, but not when `/get` is called without a forced call with `.fetch()`.\r\n\r\nIs putting the client declaration outside the handler somehow caching the database? If so, is there any way to prevent it from doing so? Are context managers required in this scenario?\r\n\r\n  [1]: https://github.com/cloudant/python-cloudant"},"comment":{"url":"https://api.github.com/repos/cloudant/python-cloudant/issues/comments/388793763","html_url":"https://github.com/cloudant/python-cloudant/issues/380#issuecomment-388793763","issue_url":"https://api.github.com/repos/cloudant/python-cloudant/issues/380","id":388793763,"user":{"login":"ricellis","id":10698320,"avatar_url":"https://avatars0.githubusercontent.com/u/10698320?v=4","gravatar_id":"","url":"https://api.github.com/users/ricellis","html_url":"https://github.com/ricellis","followers_url":"https://api.github.com/users/ricellis/followers","following_url":"https://api.github.com/users/ricellis/following{/other_user}","gists_url":"https://api.github.com/users/ricellis/gists{/gist_id}","starred_url":"https://api.github.com/users/ricellis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ricellis/subscriptions","organizations_url":"https://api.github.com/users/ricellis/orgs","repos_url":"https://api.github.com/users/ricellis/repos","events_url":"https://api.github.com/users/ricellis/events{/privacy}","received_events_url":"https://api.github.com/users/ricellis/received_events","type":"User","site_admin":false},"created_at":"2018-05-14T12:06:14Z","updated_at":"2018-05-14T12:06:14Z","author_association":"CONTRIBUTOR","body":"As @tomblench said \"Documents are locally cached as a feature\".\r\nCalling `save()` updates the remote db and then updates the local dict copy of the document with the new `_rev`. That is the intended behaviour and the behaviour I am able to reproduce locally both with or without a context manager.\r\nThe `db[document_id]` will `fetch()` if the document is not present in the local dict, but will not update the dict if there is already a local copy.\r\n\r\nThus the behaviour I would expect for your scenario (and what I see locally) is:\r\n1. Call update\r\n    1. Remote call to check for DB\r\n    1. Add DB to local dict\r\n    1. Remote call to fetch document\r\n    1. Add document to local dict (rev 1-xxxx)\r\n    1. Document content updated\r\n    1. Remote call to update document (makes rev 2-xxxx)\r\n    1. Local dict `_rev` updated (now rev 2-xxxx)\r\n1. Call get\r\n    1. Local dict DB is returned\r\n    1. Local dict document is returned (rev 2-xxxx)\r\n1. Call update\r\n    1. Local dict DB is returned\r\n    1. Local dict document is returned (rev 2-xxxx)\r\n    1. Document content updated\r\n    1. Remote call to update document (makes rev 3-xxxx)\r\n    1. Local dict `_rev updated` (now rev 3-xxxx)\r\n1. Call get\r\n    1. Local dict DB is returned\r\n    1. Local dict document is returned (rev 3-xxxx)\r\n\r\nGiven that you don't see that behaviour my guess is that your serverless env is re-using the container for some calls and using a new container for others. For example the behaviour you see could be caused by the client/dict combination used to do the first `update` or first `get` being re-used for the second `get` (i.e. with the second `update` happening in a different container instance and so the local update to `_rev 3-xxxx` happens in a completely different dict instance). IIUC [AWS Lambda, for example,](https://aws.amazon.com/blogs/compute/container-reuse-in-lambda/) is at liberty to re-use containers. I'm not sure how deep that re-use goes and what level of re-initialization of objects etc happens, but that's an environment thing that we wouldn't debug.\r\n\r\nDoing an explicit fetch (or having the context manager perform that automatically) ensures the state of the local dict is \"current\" in whatever container you're in before you get the document from it. There is currently no way to disable the local caching in python-cloudant."}}
{"actor":{"display_login":"ricellis"}}