{"push_id":1351339519,"size":1,"distinct_size":1,"ref":"refs/heads/mem-pool","head":"1a2b74f31fcfe9101450279c36d55bf856742211","before":"bbab8ce7620544d27e5c7a165d9cd715276e94aa","commits":[{"sha":"1a2b74f31fcfe9101450279c36d55bf856742211","author":{"name":"Raphael 'kena' Poss","email":"7541742341763867a641aada528529ea551ae635@cockroachlabs.com"},"message":"sql: implement a memory policy that prevents server crashes for large queries.\n\nPrior to this patch, any query that required rows to be loaded in\nserver memory prior to being sent back to the client could cause the\nserver to crash if there is insufficient RAM to store all rows.\n\nThis patch builds upon the `MemoryUsageMonitor` abstraction (now\nrenamed to `MemoryMonitor`) by enabling chaining/stacking of monitors\nupon each other.\n\nA single `MemoryMonitor` object is shared by all client connections over\npgwire.  Its capacity for a given node is 25% of total node RAM size,\nunless overridden via `--sql-memory` on that node's `start` command\nline.\n\nEach client connection starts with a budget of 1KiB (configurable via\nthe env. var. `COCKROACH_BASE_SQL_MEMORY_BUDGET`). This budget is\npre-reserved from the common pool by the pgwire server (using a\npgwire-wide `MemoryMonitor`), and reused when a connection is closed\nwithout interacting with the pool. This ensures that the pool is not\nreached at all for what we expect is common usage: many clients using\nonly OLTP queries returning very few rows (or none) and without need\nfor dynamic server-side row storage.\n\nOnce a client connection exceeds its pre-reserved budget, its\nsession-bound \"top\" `MemoryMonitor` starts interacting with the pool\nby requesting fixed-size chunks. These chunks are 10KiB by default,\nand this size is configurable via the\nenv. var. `COCKROACH_MEMORY_ALLOCATION_CHUNK_SIZE`. If a session is\nlong-lived, a monitor will relinquish unused memory to the common pool\nwhen there are more than 10 unused reserved chunks (this factor is\nconfigurable via `COCKROACH_MAX_ALLOCATED_UNUSED_BLOCKS`).\n\nWithin a session two sub-monitors further detail allocations.  The\n\"session monitor\" tracks allocations that can span an entire session,\nfor example result sets and prepared statements. The \"txn monitor\"\ntracks allocations within transactions.\n\nA separate stack of `MemoryMonitor` objects supports SQL allocations\nby the `adminServer` for UI API calls and admin commands; this is\ncurrently unbounded.\n\nAnother separate stack of monitors also supports allocations by the\ninternal executor (SQL leases, log events, etc.), also unbounded.\n\nThe various monitor statistics are also aggregated in the admin UI.","distinct":true,"url":"https://api.github.com/repos/knz/cockroach/commits/1a2b74f31fcfe9101450279c36d55bf856742211"}]}
{"actor":{"display_login":"knz"}}