{"push_id":3062760779,"size":27,"distinct_size":3,"ref":"refs/heads/non_blocking_log_read","head":"bbfe187c84d3278538056b229e8ef2773ecb0c2c","before":"d0d668f427fd3422b5dd04214dbc4d7a66610ea3","commits":[{"sha":"0f88160f67d36ea30e3aca3a3cef924c3a009be6","author":{"name":"DorianZheng","email":"865d747c6044f27c952d5cad7e930eb52bca83fd@gmail.com"},"message":"Fix `CompactFiles` bug (#4665)\n\nSummary:\n`CompactFiles` gets `SuperVersion` before `WaitForIngestFile`, while `IngestExternalFile` may add files that overlap with `input_file_names`\n\nThe timeline of execution flow is as follow:\n\nLet's say that level N has two file [1,2] and [5,6]\n```\ntimeline              user_thread1                             user_thread2\nt0   |      CompactFiles([1, 2], [5, 6]) begin\nt1   |         GetReferencedSuperVersion()\nt2   |                                              IngestExternalFile([3,4]) to level N begin\nt3   |             CompactFiles resume\n     V\n```\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4665\n\nDifferential Revision: D13030674\n\nPulled By: ajkr\n\nfbshipit-source-id: 8be19477fd6e505032267a979d32f3097cc3be51","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/0f88160f67d36ea30e3aca3a3cef924c3a009be6"},{"sha":"050d73551b7f0f02f8d9f14ff52fcdf2ea7eba72","author":{"name":"Fosco Marotto","email":"e06c5ff635d94155802c5a15105ffae4c587024b@fb.com"},"message":"Update history and version for future 5.18 release.\n\nSummary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/4669\n\nDifferential Revision: D13031522\n\nPulled By: gfosco\n\nfbshipit-source-id: d09e655a9d5556594f195c5d1b786900932145ce","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/050d73551b7f0f02f8d9f14ff52fcdf2ea7eba72"},{"sha":"7d04ef465590263cedec0219a38acf96116b1605","author":{"name":"Abhishek Madan","email":"04d4c10587cc0667faca947d04760f909bfc332d@fb.com"},"message":"Fix flaky DBDynamicLevelTest.DynamicLevelMaxBytesBase2 (#4668)\n\nSummary:\nPart of the test required that a compaction start before a\nmanual flush, but this was not enforced by the test. In some cases,\nparticularly when writing to tmpfs, this could lead to the compaction\nstarting after the flush, which caused the base level to be higher than\nit was expected to be. Add a sync point in the test to ensure that the\nflush and compaction happen simultaneously.\n\nThe test also had some stale comments, so those have been removed or\nmodified, and the test has been simplified so that it no longer uses sleeps\nand writes uncompressed SSTs.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4668\n\nDifferential Revision: D13032440\n\nPulled By: abhimadan\n\nfbshipit-source-id: 3f23b583a096454dafb8d8ea75678605dec80209","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/7d04ef465590263cedec0219a38acf96116b1605"},{"sha":"dd742e2416eadff64c330e3b72b4aac009c25100","author":{"name":"Sagar Vemuri","email":"3ced6d38c5f829e958d8e628fbea488f7e956d44@fb.com"},"message":"Automatically set LITE=1 on passing OPT=\"-DROCKSDB_LITE\" (#4671)\n\nSummary:\nIn #4652 we are setting -Os for lite builds only when LITE=1 is specified. But currently almost all the users invoke lite build via OPT=\"-DROCKSDB_LITE=1\". So this diff tries to set LITE=1 when users already pass in -DROCKSDB_LITE=1 via the command line.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4671\n\nDifferential Revision: D13033801\n\nPulled By: sagar0\n\nfbshipit-source-id: e7b506cee574f9e3f42221ee6647915011c78d78","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/dd742e2416eadff64c330e3b72b4aac009c25100"},{"sha":"2993cd2002cb7b45143d53ecde6ff912095d8eb6","author":{"name":"Sagar Vemuri","email":"3ced6d38c5f829e958d8e628fbea488f7e956d44@fb.com"},"message":"Fix RocksDB Lite build (#4675)\n\nSummary:\nOur internal CI test caught RocksDB Lite build failures. The failures are due to a new test introduced in #4665 using `SSTFileWriter` and `IngestExternalFile`, but these is not exposed under lite mode. Fixed by #ifdef'ing out the test.\n\n```\ndb/db_test2.cc: In member function ‘virtual void rocksdb::DBTest2_TestCompactFiles_Test::TestBody()’:\ndb/db_test2.cc:2907:3: error: ‘SstFileWriter’ is not a member of ‘rocksdb’\n   rocksdb::SstFileWriter sst_file_writer{rocksdb::EnvOptions(), options};\n   ^\nIn file included from ./util/testharness.h:15:0,\n                 from ./table/mock_table.h:23,\n                 from ./db/db_test_util.h:44,\n                 from db/db_test2.cc:13:\ndb/db_test2.cc:2912:13: error: ‘sst_file_writer’ was not declared in this scope\n   ASSERT_OK(sst_file_writer.Open(external_file1));\n```\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4675\n\nDifferential Revision: D13035984\n\nPulled By: sagar0\n\nfbshipit-source-id: c1ceac550dfac1a85eeea436693dc7dd467519a6","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/2993cd2002cb7b45143d53ecde6ff912095d8eb6"},{"sha":"b3130193266aa732310e094f507e81632eff092d","author":{"name":"Zhongyi Xie","email":"3abcc4d56e54c30e8e7aa72e11e8c6aa39738f74@fb.com"},"message":"use per-level perfcontext for DB::Get calls (#4617)\n\nSummary:\nthis PR adds two more per-level perf context counters to track\n* number of keys returned in Get call, break down by levels\n* total processing time at each level during Get call\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4617\n\nDifferential Revision: D12898024\n\nPulled By: miasantreble\n\nfbshipit-source-id: 6b84ef1c8097c0d9e97bee1a774958f56ab4a6c4","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/b3130193266aa732310e094f507e81632eff092d"},{"sha":"ea9454700a65c0a64b8f401e74f2ba82b032dd2e","author":{"name":"Andrew Kryczka","email":"0323c12ab25c67b2577c7e6072f00792cdc36098@fb.com"},"message":"Backup engine support for direct I/O reads (#4640)\n\nSummary:\nUse the `DBOptions` that the backup engine already holds to figure out the right `EnvOptions` to use when reading the DB files. This means that, if a user opened a DB instance with `use_direct_reads=true`, then using `BackupEngine` to back up that DB instance will use direct I/O to read files when calculating checksums and copying. Currently the WALs and manifests would still be read using buffered I/O to prevent mixing direct I/O reads with concurrent buffered I/O writes.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4640\n\nDifferential Revision: D13015268\n\nPulled By: ajkr\n\nfbshipit-source-id: 77006ad6f3e00ce58374ca4793b785eea0db6269","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/ea9454700a65c0a64b8f401e74f2ba82b032dd2e"},{"sha":"a47868226087aebd87bc9d35104c7384c6cc7b00","author":{"name":"Soli","email":"ff2b5f2ccabd6cb198e9935ff53b200116be55a0@cbug.org"},"message":"Fix #3840: only `SyncClosedLogs` for multiple CFs (#4460)\n\nSummary:\nCall `SyncClosedLogs()` only if there are more than one column families.\n\nUpdate several unit tests (in `fault_injection_test` and `db_flush_test`) correspondingly.\n\nSee #3840 for more info.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4460\n\nDifferential Revision: D12896377\n\nPulled By: riversand963\n\nfbshipit-source-id: f49afdaec32568f12f001219a3aec1dfde3b32bf","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/a47868226087aebd87bc9d35104c7384c6cc7b00"},{"sha":"5945e16dfc6e99522c607841cfd387eebed256fc","author":{"name":"Soli Como","email":"ff2b5f2ccabd6cb198e9935ff53b200116be55a0@cbug.org"},"message":"Divide `NO_ITERATORS` into two counters `NO_ITERATOR_CREATED` and `NO_ITERATOR_DELETE` (#4498)\n\nSummary:\nCurrently, `Statistics` can record tick by `recordTick()` whose second parameter is an `uint64_t`.\nThat means tick can only increase.\nIf we want to reduce tick, we have to work around like `RecordTick(statistics_, NO_ITERATORS, uint64_t(-1));`.\nThat's kind of a hack.\n\nSo, this PR divide `NO_ITERATORS` into two counters `NO_ITERATOR_CREATED` and `NO_ITERATOR_DELETE`, making the counters increase only.\n\nFixes #3013 .\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4498\n\nDifferential Revision: D10395010\n\nPulled By: sagar0\n\nfbshipit-source-id: cfb523b22a37411c794b4e9da090f1ae30293db2","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/5945e16dfc6e99522c607841cfd387eebed256fc"},{"sha":"4f0fcb78ae685e80527da2034d7101b64d23fa2d","author":{"name":"QingpingWang","email":"79fc0544fcc0fa6155f1676ccf378ecda45103a5@gmail.com"},"message":"Expose num entries and deletions of sst files (#4623)\n\nSummary:\nhe ratio of num_deletions to num_entries of a level can be useful to determine if a manual compaction needs to be triggered on a level.\nAlso refer #3980\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4623\n\nDifferential Revision: D13045744\n\nPulled By: sagar0\n\nfbshipit-source-id: 71f3c8e363a8ffd194ec3bb0ed0b69612231f0b3","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/4f0fcb78ae685e80527da2034d7101b64d23fa2d"},{"sha":"05dab3aacd877ca6ef500ac1f46efbbd0774cb27","author":{"name":"Yi Wu","email":"bc6ae85de42d744f2dc8adc7731d42b35611b18f@fb.com"},"message":"BlobDB: use char array instead of string as buffer (#4662)\n\nSummary:\nAs pointed out in #4059, we miss use string as buffer for file read. Changing to use char array instead.\n\nClosing #4059\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4662\n\nDifferential Revision: D13012998\n\nPulled By: yiwu-arbug\n\nfbshipit-source-id: 41234ba17c0bccea65bd647e362a0e979152bd1e","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/05dab3aacd877ca6ef500ac1f46efbbd0774cb27"},{"sha":"abb1a8fc23755ce6861f4b88f528068f84ea337f","author":{"name":"Siying Dong","email":"485e2df9581649b6c8f873e000ad8bee4290c1b7@fb.com"},"message":"Add a unit test to assert number of preads (#4657)\n\nSummary:\nWe used to have a bug, which caused every block to be read twice, and none of our tests caught it. Add a very simply unit test to make sure that when reading a data block, we only issue one pread against the SST file.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4657\n\nDifferential Revision: D13005260\n\nPulled By: siying\n\nfbshipit-source-id: 03167b554ad2451192b1707415536d7d05e9026c","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/abb1a8fc23755ce6861f4b88f528068f84ea337f"},{"sha":"b32d087dbb3b9d9f2c9597caa650d0ca9d2e2d7f","author":{"name":"Yi Wu","email":"bc6ae85de42d744f2dc8adc7731d42b35611b18f@fb.com"},"message":"Move MemoryAllocator option from Cache to BlockBasedTableOptions (#4676)\n\nSummary:\nPer offline discussion with siying, `MemoryAllocator` and `Cache` should be decouple. The idea is that memory allocator handles memory allocation, while cache handle cache policy.\n\nIt is normal that external cache libraries pack couple the two components for better optimization. If we want to integrate with such library in the future, we can make a wrapper of the library implementing both `Cache` and `MemoryAllocator` interface.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4676\n\nDifferential Revision: D13047662\n\nPulled By: yiwu-arbug\n\nfbshipit-source-id: cd42e246d80ab600b4de47d073f7d2db308ce6dd","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/b32d087dbb3b9d9f2c9597caa650d0ca9d2e2d7f"},{"sha":"a2de8e52bb6c13baf5f2323eba0ca356f1294f88","author":{"name":"Simon Liu","email":"36a167f46d6530a6957d4c869c3a79f0743ddfac@gmail.com"},"message":"optimized the performance of autovector::emplace_back. (#4606)\n\nSummary:\nIt called the autovector::push_back simply in autovector::emplace_back.\nThis was not efficient, and then optimazed this function through the\nperfect forwarding.\n\nThis was the src and result of the benchmark(using the google'benchmark library, the type of elem in\nautovector was std::string, and call emplace_back with the \"char *\" type):\n\nhttps://gist.github.com/monadbobo/93448b89a42737b08cbada81de75c5cd\n\nPS: The benchmark's result of  previous PR was not accurate, and so I update the test case and result.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4606\n\nDifferential Revision: D13046813\n\nPulled By: sagar0\n\nfbshipit-source-id: 19cde1bcadafe899aa454b703acb35737a1cc02d","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/a2de8e52bb6c13baf5f2323eba0ca356f1294f88"},{"sha":"b76398a82bde58bfcfa3ed5ba3dbfb6168c241de","author":{"name":"Zhongyi Xie","email":"3abcc4d56e54c30e8e7aa72e11e8c6aa39738f74@fb.com"},"message":"apply ReadOptions.iterate_upper_bound to transaction iterator (#4656)\n\nSummary:\nCurrently transaction iterator does not apply `ReadOptions.iterate_upper_bound` when iterating. This PR attempts to fix the problem by having `BaseDeltaIterator` enforcing the upper bound check when iterator state is changed.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4656\n\nDifferential Revision: D13039257\n\nPulled By: miasantreble\n\nfbshipit-source-id: 909eb9f6b4597a4d80418fb139f32ec82c6ec1d1","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/b76398a82bde58bfcfa3ed5ba3dbfb6168c241de"},{"sha":"b82e57d42558d62394c54d41990177e1ca87085d","author":{"name":"Siying Dong","email":"485e2df9581649b6c8f873e000ad8bee4290c1b7@fb.com"},"message":"Remove two variables from BlockContents class and don't use class Block for compressed block (#4650)\n\nSummary:\nWe carry compression type and \"cachable\" variables for every block in the block cache, while they take well-known values. 8-byte is wasted for each block (2-byte for useful information but it takes 8 bytes because of padding). With this change, these two variables are removed.\n\nThe cachable information is only useful in the process of reading the block. We use other information to infer from it. For compressed blocks, the compression type is a part of the block content itself so we can get it from there.\n\nSome code is slightly refactored so that the cachable information can flow better.\n\nAnother change is to only use class BlockContents for compressed block, and narrow the class Block to only be used for uncompressed blocks, including blocks in compressed block cache. This can make the Block class less confusing. It also saves tens of bytes for each block in compressed block cache.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4650\n\nDifferential Revision: D12969070\n\nPulled By: siying\n\nfbshipit-source-id: 548b62724e9eb66993026429fd9c7c3acd1f95ed","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/b82e57d42558d62394c54d41990177e1ca87085d"},{"sha":"d8df169b8498609eda28c0d6c2d91588b0aa925b","author":{"name":"Zhongyi Xie","email":"3abcc4d56e54c30e8e7aa72e11e8c6aa39738f74@fb.com"},"message":"release db mutex when calling ApproximateSize (#4630)\n\nSummary:\n`GenSubcompactionBoundaries` calls `VersionSet::ApproximateSize` which gets BlockBasedTableReader for every file and seeks in its index block to find `key`'s offset. If the table or index block aren't in memory already, this involves I/O. This can be improved by releasing DB mutex when calling ApproximateSize.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4630\n\nDifferential Revision: D13052653\n\nPulled By: miasantreble\n\nfbshipit-source-id: cae31d46d10d0860fa8a26b8d5154b2d17d1685f","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/d8df169b8498609eda28c0d6c2d91588b0aa925b"},{"sha":"5cf5f4724d5816022249a82ea773328579a56b51","author":{"name":"thisisbaozi","email":"4a71a0654297493841a2ba043a72c0157142553d@gmail.com"},"message":"Expose underlying Read/Write APIs for avoiding unnecessary memory copy (#2303)\n\nSummary:\nadamretter\n\nAs you already mentioned at #1247 .\nPull Request resolved: https://github.com/facebook/rocksdb/pull/2303\n\nDifferential Revision: D10209001\n\nPulled By: sagar0\n\nfbshipit-source-id: bcbce004112c2edeaff116968d79c6f90aab4b6c","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/5cf5f4724d5816022249a82ea773328579a56b51"},{"sha":"de651035535b6d66276edc97c121cb507e9138cb","author":{"name":"Yanqin Jin","email":"c238530decc239d4bf11aaf1a7c1c1cd8e3d0095@fb.com"},"message":"Improve result report of scan (#4648)\n\nSummary:\nWhen iterator becomes invalid, there are two possibilities.\nFirst, all data in the column family have been scanned and there is nothing\nmore to scan.\nSecond, an underlying error has occurred, causing `status()` to be !ok.\nTherefore, we need to check for both cases when `!iter->Valid()`.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4648\n\nDifferential Revision: D12959601\n\nPulled By: riversand963\n\nfbshipit-source-id: 49c9382c9ea9e78f2e2b6f3708f0670b822ca8dd","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/de651035535b6d66276edc97c121cb507e9138cb"},{"sha":"0ed738fdd0f3b912dd6eca75dfc203d2d1f5ca31","author":{"name":"Abhishek Madan","email":"04d4c10587cc0667faca947d04760f909bfc332d@fb.com"},"message":"Add max_scan_distance flag to db_bench (#4660)\n\nSummary:\nThe new flag makes it possible to constrain iterator traversal\nby the upper/lower bound the iterator is expected to pass. This allows\nseekrandom results to be more easily comparable between DBs with and\nwithout deletions.\nPull Request resolved: https://github.com/facebook/rocksdb/pull/4660\n\nDifferential Revision: D13053111\n\nPulled By: abhimadan\n\nfbshipit-source-id: 33e250f2e2d210b54c7726399da30a33f723c33c","distinct":false,"url":"https://api.github.com/repos/riversand963/rocksdb/commits/0ed738fdd0f3b912dd6eca75dfc203d2d1f5ca31"}]}
{"actor":{"display_login":"riversand963"}}