{"push_id":3183667807,"size":65,"distinct_size":1,"ref":"refs/heads/KAFKA-7499","head":"cce297fafc2c80b10111799add780e5f78647981","before":"14822eb0fb8ee4dfa1128c4f77200a2a0b589a89","commits":[{"sha":"ab1fb3fdde95be535b97eb1c8c21ef32896f891a","author":{"name":"Bill Bejeck","email":"5c404be01b63927a28df603569da1321e84406a6@gmail.com"},"message":"MINOR: Adding system test for named repartition topics (#5913)\n\nThis is a system test for doing a rolling upgrade of a topology with a named repartition topic.\r\n\r\n1. An initial Kafka Streams application is started on 3 nodes. The topology has one operation forcing a repartition and the repartition topic is explicitly named.\r\n2. Each node is started and processing of data is validated\r\n3. Then one node is stopped (full stop is verified)\r\n4. A property is set signaling the node to add operations to the topology before the repartition node which forces a renumbering of all operators (except repartition node)\r\n5. Restart the node and confirm processing records\r\n6. Repeat the steps for the other 2 nodes completing the rolling upgrade\r\n\r\nI ran two runs of the system test with 25 repeats in each run for a total of 50 test runs.\r\nAll test runs passed\r\n\r\nReviewers: John Roesler <john@confluent.io>, Matthias J. Sax <matthias@confluent.io>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/ab1fb3fdde95be535b97eb1c8c21ef32896f891a"},{"sha":"b4030a0375a46f71000d11147192db83a7f3bb25","author":{"name":"Jakub Scholz","email":"c50267b906a652f2142cfab006e215c9f6fdc8a0@scholzj.com"},"message":"MINOR: Update command line options in Authorization and ACLs documentation chapter (#5995)","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/b4030a0375a46f71000d11147192db83a7f3bb25"},{"sha":"7283711c0d99d484b10dd7c460f7df01a319fc2b","author":{"name":"Srinivas Reddy","email":"c494f431fabd734a11bb48b02d45b46c9efcb356@users.noreply.github.com"},"message":"KAFKA-7446: Fix the duration and instant validation messages. (#5930)\n\nChanges made as part of this commit.\r\n - Improved error message for better readability at millis validation utility\r\n - Corrected java documentation on `AdvanceInterval` check.\r\n - Added caller specific prefix text to make error message more clear to developers/users.\r\n\r\nReviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Jacek Laskowski <jacek@japila.pl>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/7283711c0d99d484b10dd7c460f7df01a319fc2b"},{"sha":"437cf35373dbfa9d5c1ca44a8e2c0e363a0ffa7c","author":{"name":"Rajini Sivaram","email":"2e743c5cf650e12d7db950775eab7636545b5b8c@googlemail.com"},"message":"KAFKA-7702: Fix matching of prefixed ACLs to match single char prefix (#5994)\n\nReviewers: Jun Rao <junrao@gmail.com>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/437cf35373dbfa9d5c1ca44a8e2c0e363a0ffa7c"},{"sha":"f65b1c4796fdf34a883e0228f7b74a0fe36dd4a8","author":{"name":"huxi","email":"dd5c486cc5d693c862549ee26ac47a98670983f6@hotmail.com"},"message":"KAFKA-7687; Print batch level information in DumpLogSegments when deep iterating (#5976)\n\nDumpLogSegments should print batch level information when deep-iteration is specified.\r\n\r\nReviewers: Jason Gustafson <jason@confluent.io>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/f65b1c4796fdf34a883e0228f7b74a0fe36dd4a8"},{"sha":"a1807d491450480c32d86df2df3f8b685c65e6ed","author":{"name":"Matthias J. Sax","email":"b36a4b8a3b519984e26e470684cfa9887ab74e9b@apache.org"},"message":"MINOR: Improve GlobalKTable docs (#5996)\n\nReviewers: Jim Galasyn, Michael G. Noll, John Roesler, Bill Bejeck, Guozhang Wang","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/a1807d491450480c32d86df2df3f8b685c65e6ed"},{"sha":"0d4cf64af359d22749f7e865c4efaee773d64962","author":{"name":"Rajini Sivaram","email":"2e743c5cf650e12d7db950775eab7636545b5b8c@googlemail.com"},"message":"KAFKA-7697: Process DelayedFetch without holding leaderIsrUpdateLock (#5999)\n\nDelayed fetch operations acquire leaderIsrUpdate read lock of one or more Partitions from the fetch request when attempting to complete the fetch operation. While appending new records, complete fetch requests after releasing leaderIsrUpdate of the Partition to which records were appended to avoid deadlocks in request handler threads.\r\n\r\nReviewers: Jason Gustafson <jason@confluent.io>, Jun Rao <junrao@gmail.com>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/0d4cf64af359d22749f7e865c4efaee773d64962"},{"sha":"ec501f305e53a09072580fb3824048c170d32a48","author":{"name":"Nikolay","email":"73fd4527cc1ec76b3a1337424296716ad5e4a002@apache.org"},"message":"KAFKA-7420: Global store surrounded by read only implementation (#5865)\n\nReviewers: Matthias J. Sax <matthias@confluent.io>, Kamal Chandraprakash (@kamalcph), Bill Bejeck <bill@confluent.io>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/ec501f305e53a09072580fb3824048c170d32a48"},{"sha":"b616f913c8a57609476e5cf99b1802ef1dfe1c98","author":{"name":"Jonathan Santilli","email":"81464ccc366e8cae29999c27e67b273b3da70fbc@users.noreply.github.com"},"message":"KAFKA-7678: Avoid NPE when closing the RecordCollector (#5993)\n\nReviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/b616f913c8a57609476e5cf99b1802ef1dfe1c98"},{"sha":"9f954ac614cd5dd7efbcabe34799207128f16e63","author":{"name":"Cyrus Vafadari","email":"22db70456c0d6f279c1593486822093ade9d891b@confluent.io"},"message":"MINOR: Safe string conversion to avoid NPEs\n\nShould be ported back to 2.0\n\nAuthor: Cyrus Vafadari <cyrus@confluent.io>\n\nReviewers: Ewen Cheslack-Postava <ewen@confluent.io>\n\nCloses #6004 from cyrusv/cyrus-npe","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/9f954ac614cd5dd7efbcabe34799207128f16e63"},{"sha":"0ffdf8307f8c0822def6f97f69777337106fc347","author":{"name":"Anna Povzner","email":"2bc1ecb410e142bce83bce6f212b41e1781536dc@confluent.io"},"message":"KAFKA-6388; Recover from rolling an empty segment that already exists (#5986)\n\nThere were several reported incidents where the log is rolled to a new segment with the same base offset as an active segment, causing KafkaException: Trying to roll a new log segment for topic partition X-N with start offset M while it already exists. In the cases we have seen, this happens to an empty log segment where there is long idle time before the next append and somehow we get to a state where offsetIndex.isFull() returns true due to _maxEntries == 0. This PR recovers from this state by deleting and recreating the segment and all of its associated index files.\r\n\r\nReviewers: Jason Gustafson <jason@confluent.io>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/0ffdf8307f8c0822def6f97f69777337106fc347"},{"sha":"38e7d5763fda21608d1b29742cf502b257bef157","author":{"name":"Guozhang Wang","email":"fae442ba222671b2ebdf7511c9ca77079c43031b@gmail.com"},"message":" KAFKA-7673: Upgrade rocksdb to 5.15.10 (#5985)\n\nReviewers: Matthias J. Sax <mjsax@apache.org>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/38e7d5763fda21608d1b29742cf502b257bef157"},{"sha":"87cc31c4e7ea36e7e832a1d02d71480a91a75293","author":{"name":"huxi","email":"dd5c486cc5d693c862549ee26ac47a98670983f6@hotmail.com"},"message":"KAFKA-7704: MaxLag.Replica metric is reported incorrectly (#5998)\n\nOn the follower side, for the empty `LogAppendInfo` retrieved from the leader, fetcherLagStats set the wrong lag for fetcherLagStats due to `nextOffset` is zero.","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/87cc31c4e7ea36e7e832a1d02d71480a91a75293"},{"sha":"c05050346468cc27bcfb3a43bde90c47533a386c","author":{"name":"Mark Cho","email":"0a96bcf52d5799c46c4187f0e3f57d9b5da11a7c@gmail.com"},"message":"KAFKA-7709: Fix ConcurrentModificationException when retrieving expired inflight batches on multiple partitions. (#6005)\n\nReviewers: Dhruvil Shah <dhruvil@confluent.io>, Guozhang Wang <wangguoz@gmail.com>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/c05050346468cc27bcfb3a43bde90c47533a386c"},{"sha":"05dc36d548fbe8e00dff51b19d698c980e7ecf0b","author":{"name":"Stanislav Kozlovski","email":"985edd36ffb4acfa9d96704f76f324aa106f51b7@outlook.com"},"message":"MINOR: Fix failing ConsumeBenchTest:test_multiple_consumers_specified_group_partitions_should_raise (#6015)\n\nThis is the error message we're after:\r\n\r\n\"You may not specify an explicit partition assignment when using multiple consumers in the same group.\"\r\n\r\nWe apparently changed it midway through #5810 and forgot to update the test.\r\n\r\nReviewers: Dhruvil Shah <dhruvil@confluent.io>, Ismael Juma <ismael@juma.me.uk>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/05dc36d548fbe8e00dff51b19d698c980e7ecf0b"},{"sha":"ac35ef6242d6893223af946f3c9438b8b4294389","author":{"name":"Gardner Vickers","email":"872acb7ab7600f079c7fd2865a15af11a8d38519@vickers.me"},"message":"MINOR: Specify character encoding in NetworkTestUtils (#5965)\n\nThis attempts to address the flaky test `SaslAuthenticatorTest.testCannotReauthenticateWithDifferentPrincipal()` \r\n\r\nI was not able to reproduce locally even after 150 test runs in a loop, but given the error message:\r\n\r\n```\r\norg.junit.ComparisonFailure: expected:\r\n<[6QBJiMZ6o5AqbNAjDTDjWtQSa4alfuUWsYKIy2tt7dz5heDaWZlz21yr8Gl4uEJkQABQXeEL0UebdpufDb5k8SvReSK6wYwQ9huP-9]> but was:<[????ï¿½ï¿½ï¿½ï¿½????OAUTHBEARER]>\r\n```\r\n\r\n`????ï¿½ï¿½ï¿½ï¿½????` seems to mean invalid UTF-8.\r\n\r\nWe now specify the charset when writing out and reading in bytes.\r\n\r\nReviewers: Ismael Juma <ismael@juma.me.uk>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/ac35ef6242d6893223af946f3c9438b8b4294389"},{"sha":"edfa68173615cddaa421ef02ccf7b82ce4181c79","author":{"name":"Stanislav Kozlovski","email":"985edd36ffb4acfa9d96704f76f324aa106f51b7@outlook.com"},"message":"MINOR: Catch NoRecordsException in testCommaSeparatedRegex() test (#5944)\n\nThis test sometimes fails with\r\n\r\n```\r\nkafka.tools.MirrorMaker$NoRecordsException\r\n\tat kafka.tools.MirrorMaker$ConsumerWrapper.receive(MirrorMaker.scala:483)\r\n\tat kafka.tools.MirrorMakerIntegrationTest$$anonfun$testCommaSeparatedRegex$1.apply$mcZ$sp(MirrorMakerIntegrationTest.scala:92)\r\n\tat kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:738)\r\n```\r\n\r\nThe test should catch `NoRecordsException` instead of `TimeoutException`.\r\n\r\nReviewers: Ismael Juma <ismael@juma.me.uk>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/edfa68173615cddaa421ef02ccf7b82ce4181c79"},{"sha":"b49229675748afae584799e11a8b8f090c4218a9","author":{"name":"John Roesler","email":"884ff5c4006c6f627a23dfdf6dcce23badd4698e@users.noreply.github.com"},"message":"MINOR: fix checkpoint write failure warning log (#6008)\n\nWe saw a log statement in which the cause of the failure to write a checkpoint was not properly logged.\r\nThis change logs the exception properly and also verifies the log message.\r\n\r\nReviewers: Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/b49229675748afae584799e11a8b8f090c4218a9"},{"sha":"c0353d8ddce88bac6fc04f85dd40cb95b8ca5cf9","author":{"name":"Guozhang Wang","email":"fae442ba222671b2ebdf7511c9ca77079c43031b@gmail.com"},"message":"KAFKA-6036: Local Materialization for Source KTable (#5779)\n\nRefactor the materialization for source KTables in the way that:\r\n\r\nIf Materialized.as(queryableName) is specified, materialize;\r\nIf the downstream operator requires to fetch from this KTable via ValueGetters, materialize;\r\nIf the downstream operator requires to send old values, materialize.\r\nOtherwise do not materialize the KTable. E.g. builder.table(\"topic\").filter().toStream().to(\"topic\") would not create any state stores.\r\n\r\nThere's a couple of minor changes along with PR as well:\r\n\r\nKTableImpl's queryableStoreName and isQueryable are merged into queryableStoreName only, and if it is null it means not queryable. As long as it is not null, it should be queryable (i.e. internally generated names will not be used any more).\r\nTo achieve this, splitted MaterializedInternal.storeName() and MaterializedInternal.queryableName(). The former can be internally generated and will not be exposed to users. QueryableName can be modified to set to the internal store name if we decide to materialize it during the DSL parsing / physical topology generation phase. And only if queryableName is specified the corresponding KTable is determined to be materialized.\r\n\r\nFound some overlapping unit tests among KTableImplTest, and KTableXXTest, removed them.\r\n\r\nThere are a few typing bugs found along the way, fixed them as well.\r\n\r\n-----------------------\r\n\r\nThis PR is an illustration of experimenting a poc towards logical materializations.\r\n\r\nToday we've logically materialized the KTable for filter / mapValues / transformValues if queryableName is not specified via Materialized, but whenever users specify queryableName we will still always materialize. My original goal is to also consider logically materialize for queryable stores, but when implementing it via a wrapped store to apply the transformations on the fly I realized it is tougher than I thought, because we not only need to support fetch or get, but also needs to support range queries, approximateNumEntries, and isOpen etc as well, which are not efficient to support. So in the end I'd suggest we still stick with the rule of always materializing if queryableName is specified, and only consider logical materialization otherwise.\r\n\r\nReviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <mjsax@apache.org>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/c0353d8ddce88bac6fc04f85dd40cb95b8ca5cf9"},{"sha":"ab156fded1548b6cf06aff56bb5193c37a8f7e51","author":{"name":"Guozhang Wang","email":"fae442ba222671b2ebdf7511c9ca77079c43031b@gmail.com"},"message":"KAFKA-6036: Follow-up; cleanup sendOldValues logic ForwardingCacheFlushListener  (#6017)\n\nThis is a follow-up PR from the previous PR #5779, where KTabeSource always get old values from the store even if sendOldValues. It gets me to make a pass over all the KTable/KStreamXXX processor to push the sendOldValues at the callers in order to avoid unnecessary store reads.\r\n\r\nMore details: ForwardingCacheFlushListener and TupleForwarder both need sendOldValues as parameters.\r\na. For ForwardingCacheFlushListener it is not needed at all, since its callers XXXCachedStore already use the sendOldValues values passed from TupleForwarder to avoid getting old values from underlying stores.\r\nb. For TupleForwarder, it actually only need to pass the boolean flag to the cached store; and then it does not need to keep it as its own variable since the cached store already respects the boolean to pass null or the actual value..\r\n\r\nThe only other minor bug I found from the pass in on KTableJoinMerge, where we always pass old values and ignores sendOldValues.\r\n\r\nReviewers: Matthias J. Sax <mjsax@apache.org>","distinct":false,"url":"https://api.github.com/repos/kamalcph/kafka/commits/ab156fded1548b6cf06aff56bb5193c37a8f7e51"}]}
{"actor":{"display_login":"kamalcph"}}