{"push_id":1143766307,"size":6,"distinct_size":5,"ref":"refs/heads/base_tid_cache","head":"9d51d5da9602b7d4363adb3ffd604f4086e05f02","before":"16d0e7d764d3d4991b91b1504856914478548a5d","commits":[{"sha":"77e731e7fe3d7c18a53eb834c1bdda24db9bfde2","author":{"name":"Vincent Pelletier","email":"f64b7451417761ab8a8a3d9cdcebbdfb1b9c6aa3@nexedi.com"},"message":"lib.locking: Control verbose locking via en environment variable.","distinct":false,"url":"https://api.github.com/repos/vpelletier/neoppod/commits/77e731e7fe3d7c18a53eb834c1bdda24db9bfde2"},{"sha":"92f342f2090b2d2c9dccedb1d322e84052c39095","author":{"name":"Vincent Pelletier","email":"f64b7451417761ab8a8a3d9cdcebbdfb1b9c6aa3@nexedi.com"},"message":"lib.protocol: Include base TID in invalidation notifications.\n\nFor now, it only allows checking that invalidated cache entry is the\nintended one.\nIt is intended to then be used to be able to process invalidations while\nthere are in-flight load calls, and accurately update their next_tid in\ncase of race-conditions, in turn allowing removal of load lock.","distinct":true,"url":"https://api.github.com/repos/vpelletier/neoppod/commits/92f342f2090b2d2c9dccedb1d322e84052c39095"},{"sha":"4caba9d26af5a079460f341ba1255eb1bcb1c9b3","author":{"name":"Vincent Pelletier","email":"f64b7451417761ab8a8a3d9cdcebbdfb1b9c6aa3@nexedi.com"},"message":"cache: Distinguish loadSerial and loadBefore cases.\n\nThis makes no difference with the current implementation, but will allow\nsmarter invalidation handling ultimately leading to load lock removal.\nAlso, inline client.app _loadFromCache into its only caller.\nAlso, extend a tiny bit cache tests.\n\nXXX: unit test failure remains, unsure of the intent:\nERROR: test_load (neo.tests.client.testClientApp.ClientApplicationTests)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File \"/home/vincent/git/neoppod/neo/tests/client/testClientApp.py\", line 224, in test_load\n    self.assertRaises(StandardError, app.load, oid, tid2)\n  File \"/usr/lib/python2.7/unittest/case.py\", line 473, in assertRaises\n    callableObj(*args, **kwargs)\n  File \"/home/vincent/git/neoppod/neo/client/app.py\", line 354, in load\n    data, tid, next_tid, _ = self._loadFromStorage(oid, tid, before_tid)\n  File \"/home/vincent/git/neoppod/neo/client/app.py\", line 377, in _loadFromStorage\n    = self._askStorage(conn, packet)\n  File \"/home/vincent/git/neoppod/neo/client/app.py\", line 191, in _askStorage\n    return self._ask(conn, packet, handler=self.storage_handler, **kw)\n  File \"/home/vincent/git/neoppod/neo/tests/client/testClientApp.py\", line 59, in _ask\n    handler.dispatch(conn, conn.fakeReceived())\nStopIteration","distinct":true,"url":"https://api.github.com/repos/vpelletier/neoppod/commits/4caba9d26af5a079460f341ba1255eb1bcb1c9b3"},{"sha":"7af0ec697184b7188fa297325054a0df8b682682","author":{"name":"Vincent Pelletier","email":"f64b7451417761ab8a8a3d9cdcebbdfb1b9c6aa3@nexedi.com"},"message":"UNUSED lib.locking: Implement a shared lock.\n\nRead locks are not mutually exclusive.\nWrite lock is exclusive.","distinct":true,"url":"https://api.github.com/repos/vpelletier/neoppod/commits/7af0ec697184b7188fa297325054a0df8b682682"},{"sha":"bcb2ff880acc939ee4d3f6b65a3cfb15657627c2","author":{"name":"Vincent Pelletier","email":"f64b7451417761ab8a8a3d9cdcebbdfb1b9c6aa3@nexedi.com"},"message":"client.app: Use \"with\" statement for cache lock and load lock.\n\nAlso, do not access _cache outside of client.app .","distinct":true,"url":"https://api.github.com/repos/vpelletier/neoppod/commits/bcb2ff880acc939ee4d3f6b65a3cfb15657627c2"},{"sha":"9d51d5da9602b7d4363adb3ffd604f4086e05f02","author":{"name":"Vincent Pelletier","email":"f64b7451417761ab8a8a3d9cdcebbdfb1b9c6aa3@nexedi.com"},"message":"WIP client.app: Remove load lock.\n\nRework the way end-tid is set on an object when such invalidation is\nreceived from master after an no-end-tid data has been received from\nstorage.\nPrevious way required strict load/load and load/tpd_finish serialisation,\nas it could only handle a single in-flight oid at any given time.\nNew way tracks all in-flight oids and merge them with any concurrent\ninvalidation before returning them to ZODB.\nThe cache lock is now used on much restricted scopes (ensuring ZODB and\nStorage caches are consistent).\nTo keep cache API clean and exception-proof, I found not other way than to\nmake cache.load fill cache on a miss (hence, trigger load from storage\nnode) via a callback.\n\nAlso, client.cache.invalidate method can be now called only once per\ninvalidation message, instead of once per invalidated oid.\n\nXXX: Do we need a commit lock, preventing any load between finish object\nvisibility on storage nodes and local caches (ZODB & Storage) update ?\nThere is already a tpc_finish/tpc_finish serialisation managed on master\nlevel for all nodes anyway, so such lock should be quite cheap.\nXXX: Cache lock may be possible to split and reduce using a shared lock for\nreads and an exclusive lock for writes - but MQ algorithm makes read alter\ncache internal structures, and it does not seem to be implemented in a\nnaturally thread-safe manner.\nTODO: rewrite cache tests.","distinct":true,"url":"https://api.github.com/repos/vpelletier/neoppod/commits/9d51d5da9602b7d4363adb3ffd604f4086e05f02"}]}
{"actor":{"display_login":"vpelletier"}}