{"push_id":3035284654,"size":18,"distinct_size":18,"ref":"refs/heads/stores_benchmark","head":"0ab94ed0c82e486dabe944ea6e10fb21cc069c6c","before":"f2ba9bdc6ec1de37e61a4920341c10eec4ab7108","commits":[{"sha":"5a0e6fbec0f35e6bcbbfb9982e547c72b8db7491","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Abstract away storage of Metric data into DataStore objects\n\nCurrently, each Metric object stores its data (labels and values) in a Hash\nobject internal to itself. This change introduces the concept of a Data Store\nthat is external to the metrics themselves, which holds all the data.\n\nThe main reason to do this is that by having a standardized and simple\ninterface that metrics use to access the store, we abstract away the details\nof storing the data from the specific needs of each metric.\n\nThis allows us to then simply swap around the stores based on the needs\nof different circumstances, with no changes to the rest of the client. The\nmain use case for this is solving the \"pre-fork\" server scenario, where\nmultiple processes need to share the metric storage, to be able to report\ncoherent numbers to Prometheus.\n\nThis change provides one store, a Synchronized hash, that pretty much has\nthe same characteristics that the existing Storage had, plus some Mutex overhead.\nWithin a single process scenario, this should be the fastest way to operate in a\nmulti-threaded environment, and the safest. As such, this is the default store.\n\nFuture commits will introduce new ones, for specific scenarios, and also each\nconsumer of the Prometheus Client can make their own and simply swap them for\nthe built-in ones, if they have specific needs.\n\nThe interface and requirements of Stores are specified in detail in a README.md\nfile in the `client/data_stores` directory. This is the documentation that must\nbe used by anyone wishing to create their own store.\n\nSigned-off-by: Daniel Magliola <danielmagliola@gocardless.com>","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/5a0e6fbec0f35e6bcbbfb9982e547c72b8db7491"},{"sha":"1e90a0ed1daa8c7eabb3069aa9fef4b2b32f91a8","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Change `text_spec.rb` to be more like an integration test\n\nThe existing spec on `Formats::Text` is based on mocking a registry with\nfake metrics and values inside, which means if anything changes in the\ninterface for metrics, the test will not catch it.\n\nAnd there's no test validating that interface.\n\nThis test is more realistic, and it actually catches the kind of bugs\nwe introduced in the process of this refactor\n\nSigned-off-by: Daniel Magliola <danielmagliola@gocardless.com>","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/1e90a0ed1daa8c7eabb3069aa9fef4b2b32f91a8"},{"sha":"6aac386a339cef2b58067c1fcf116e2c16df6972","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Add `Metric#with_labels` method\n\n`with_labels` returns a metric object, augmented with some pre-set labels.\nThis is useful to be able to reduce repetition if a certain part of the\ncode is passing the same label over and over again, but the label can\nhave different values in different parts of the codebase (so passing in\n`preset_labels` when declaring the metric is not appropriate.\n\nSigned-off-by: Daniel Magliola <danielmagliola@gocardless.com>","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/6aac386a339cef2b58067c1fcf116e2c16df6972"},{"sha":"5ceea013899d71f86e64f4f8fc8f97b68f7948eb","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Short-circuit label validation on Metric observations\n\nEvery time a metric gets observed / incremented, we are composing the\nlabels hash, merging the labels that were just passed in as part of the\nobservation with the metric's `preset_labels`, and validating that the\nlabelset is valid.\n\nIf all labels are pre-set already, however (either as `preset_labels` when\ndeclaring the metric, or by use of `with_labels`), we can validate when\nthe labels are pre-set, and then skip the validation on every observation,\nwhich can lead to some decent time savings if a metric is observed often.\n\nSigned-off-by: Daniel Magliola <danielmagliola@gocardless.com>","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/5ceea013899d71f86e64f4f8fc8f97b68f7948eb"},{"sha":"14468b07a144ec77a9c2ceddb3adb9b8ab9a67f4","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Rename confusing methods in `LabelSetValidator`\n\nLabelSetValidator has methods `valid?` and `validate`. It's not very clear\nwhat these do, what's the difference between them, and the name `valid?`\nwould imply (by convention) that it returns a Boolean, rather than raising\nan exception on invalid input.\n\nRenamed these to `validate_symbols!` and `validate_labelset!`, to make it\nclearer what each of them do. The `!` also follows the usual Ruby\nconvention of \"do X, and if that doesn't work, Raise\"\n\nThis commit also starts drawing the distinction between an array of `labels`,\nand a hash of label keys and values (which we call a `labelset`). The term\n`labels` is used interchangeably for both concepts throughout the code.\nThis commit doesn't fix all instances, but it's a step in that direction.\n\nSigned-off-by: Daniel Magliola <danielmagliola@gocardless.com>","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/14468b07a144ec77a9c2ceddb3adb9b8ab9a67f4"},{"sha":"5228d04b1be1ceb42ed7f9f859ee044c66a1b0e3","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Test \"nested\" synchronization on Data Stores\n\nThe `synchronize` method needs to be callable by consumers of a MetricStore,\nand by the MetricStore itself, when reading/writing its own data. This\ntends to lead to a \"re-entrant\" lock which is not always supported. This\ntest checks for exactly that.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/5228d04b1be1ceb42ed7f9f859ee044c66a1b0e3"},{"sha":"4771d281d8b50e6e1859343b8429139c0610661a","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Update Histogram to return buckets as strings\n\nIn the Text export format, the `le` label is reported as a string.\n\nMoreover, some of our stores may coerce Float label values into Strings.\nThis is fine, since Labelsets should be a hash of symbols to strings.\nIt also makes sense since one of those \"buckets\" will be \"+Inf\".\n\nIt's also somewhat unavoidable, since neither the Store nor the Metric\nshould be coercing those back into Floats, so report the buckets as\nstrings instead of Floats.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/4771d281d8b50e6e1859343b8429139c0610661a"},{"sha":"36518060b2077e58e81664528ef3bc4f728b4d8a","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Accumulate Histogram Buckets on Export, not Observe\n\nIn a Histogram, each bucket's count includes also the counts of all\nbuckets smaller than it.\n\nThe original code for Histograms would handle this at `observe` time,\nby incrementing all buckets bigger than the observed sample.\n\nSince calls to the Data Store may be expensive, and also all these\nincrements need to be done atomically to guarantee the export doesn't\nhave inconsistent values, this can have a big effect on performance.\n\nWith this change, at `observe` time, we only increase the bucket the\nsample falls in, and the `sum`, and do the accumulation when reading\nthe Histogram, which makes observations about 30% to 40% faster with a\n\"uniform\" distribution of sample. A sample skewed towards smaller buckets\nshould show a much larger performance gain.\n\nWith this change, we also remove the `count` key from the result of\ncalling `get`. This key is redundant (it's the same value as `+Inf`,\nand rendering this as `count` is the job of the Text Exporter already,\n`get` shouldn't be returning it.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/36518060b2077e58e81664528ef3bc4f728b4d8a"},{"sha":"812033d67b8eac6b8e2525c432d76938cb676c96","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Improve performance of Histograms\n\nMerging the `le` key into the `base_label_set` is where most time is\nspent, surprisingly.\n\nAlso somewhat surprisingly, `dup` + setting a key is faster than\n`.merge`.\n\nThis version is less pretty, but makes `observe` significantly faster\n\nFor some stores (the ones that serialize the hash into a string), we\ncould simply set the key without the `dup`, which makes this method\nabout 3x faster. However, this would require calling `dup` into all\nHash-based stores (or serializing the Hash into a string), which would\nmake THEM significantly slower.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/812033d67b8eac6b8e2525c432d76938cb676c96"},{"sha":"5edfdc581d97639a8c47332ffc630f00a5993e21","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Add `DataStores::SingleThreaded`\n\nThis is the simplest possible store we can have, which simply has a hash\nfor each metric, and no synchronization code whatsoever.\n\nIt can only be used in single-threaded, single-process scenarios, but for\nthose, it should be the fastest option.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/5edfdc581d97639a8c47332ffc630f00a5993e21"},{"sha":"3acdb686608175f5a84df84796b8a1261ab5f45e","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Add `DataStores::ConcurrentRubyHash`\n\nThis is a multi-threaded, single-process store, which makes almost no\nsynchronization efforts, and delegates almost all that trouble to\nthe `concurrent-ruby` gem, and it's \"thread-safe Hash\"\n\nIn MRI, because of the GIL, that Hash is basically just the Ruby Hash.\nAnd because of that, this store has almost the same performance\ncharacteristics as the `SingleThreaded` store. It's a lot faster than the\n`Synchronized` store.\n\nIn non-MRI, this store should be thread safe (unlike `SingleThreaded`)\nIt's not clear whether it's going to be better or equivalent to\n`Synchronized`, since they should be essentially doing roughly the same\nthing. I don't have a good way of testing, this. If you're using non-MRI,\nI suggest running the benchmark and seeing where this stands.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/3acdb686608175f5a84df84796b8a1261ab5f45e"},{"sha":"e444ef81121c275b5c19023886afcd0cb56d8638","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Add `DataStores::MMapStore`\n\nThis store keeps all its data in Memory Mapped files, one file per process\nand per metric.\nThis is generally the recommended store to use to deal with pre-fork servers\nand other \"multi-process\" scenarios.\n\nThis seems to be the fastest possible way to share data between all the\nprocesses.\n\nEach process gets their own file for a metric, which gets updated through\na mmap (so updates never make it to the actual file). When a Prometheus\nscrape is received in one of the processes, it finds all the files for\na metric, mmaps into them, reads their values and aggregates them.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/e444ef81121c275b5c19023886afcd0cb56d8638"},{"sha":"d345ea08b119597dc99be6aa7aae125bfd398523","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Add `DataStores::FileStore`\n\nThis store keeps all its data in files, through the use of PStore.\nOn every increment, the PStore gets opened, the values increased, and\nthen it gets \"Marshal dumped\", and the file saved.\n\nPStores are resilient to multi-process access (through their \"transaction\"\nlocks), so this alone solves the multi-process problem. That said, we\nhave one file per process and per thread, to keep lock contention to\na minimum, and this \"process safety\" only comes into play when exporting\nmetrics (which gathers aggregate values from all processes).\n\nWhile one would normally expect this store to be horrendously slow, it is\nactually surprisingly fast, presumably because of File System caching. The\nfiles don't seem to actually get `fsync`ed, so we're probably not really\ntouching disk.\n\nThis assumption is supported by our benchmarks, which show almost exactly\nthe same performance when using an HDD, and SSD, or `tmpfs`, which is a RAM\ndisk. The fact that this makes no difference strongly hints at the actual\ndisk not getting touched.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/d345ea08b119597dc99be6aa7aae125bfd398523"},{"sha":"052064af460de1bb195c29edc05c00ebcc7541cb","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"Add `DataStores::Redis`\n\nThis is a store that keeps all its data in Redis.\n\nThe basic idea behind using Redis is that what we need to solve the\nmulti-process problem is a shared area in memory that we can access\nin a controlled, safe, concurrent manner, and that's precisely what\nRedis is.\n\nIt's important to note that for this to work well, each server would\nneed their own local Redis server (unlike the usual configuration\nwhere there's a shared Redis server for all the App servers)\n\nNot only we don't want to share data between servers, this store can\nonly be performant if Redis is accessed through `localhost`. If we're\ninvolving an actual network, performance will tank to a probably\nunusable level.\n\nThis may be a loser compared to FileStore, but there might potentially\nbe scenarios where the store is useful, if FileStore for some reason\ncannot be used, or doesn't perform well.\n\nFor instance, on MacOs, Redis seems much faster than HFS, whereas\nin Linux, the file system is considerably faster than Redis. YMMV.","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/052064af460de1bb195c29edc05c00ebcc7541cb"},{"sha":"84e8ecdca07ccad7f45623e1ad8debb4cc505e82","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"WIP Benchmarks REWORD ME and clean me up, and add explanations to all benchmarks","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/84e8ecdca07ccad7f45623e1ad8debb4cc505e82"},{"sha":"239b9622b02bb7a445e1c2acf9711aa0d1346ef0","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"REMOVE ME!!! Fix Ruby version to 2.4.2","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/239b9622b02bb7a445e1c2acf9711aa0d1346ef0"},{"sha":"2f66d831553219dbfbdffa7e2b08e0c28c53ac10","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"XXXX Central Hash Example, Kill this crap","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/2f66d831553219dbfbdffa7e2b08e0c28c53ac10"},{"sha":"0ab94ed0c82e486dabe944ea6e10fb21cc069c6c","author":{"name":"Daniel Magliola","email":"89d49b6744ccc763daa8041f77a567a08347f890@gocardless.com"},"message":"WIPWIPWIP Undo me and Finish Me","distinct":true,"url":"https://api.github.com/repos/gocardless/prometheus_client_ruby/commits/0ab94ed0c82e486dabe944ea6e10fb21cc069c6c"}]}
{"actor":{"display_login":"dmagliola"}}