Repository navigation
Conversation
Merging items at levels where both sketches hold items copied from other after this sketch's items had been moved out, so a throwing copy left destroyed items counted (a double free). A weighted update reaches this once the sketch is in estimation mode. - populate_work_arrays() copies (or moves) other's items first; if a copy throws, it destroys them and this sketch is unchanged. Then this sketch's items are merged in front of them with moves only. - update() and merge() construct an item in level zero before counting it, and reset the sorted view before any change. - The copy and converting constructors release their items when a copy or the sort check throws. - The KLL helpers use the sketch's comparator instead of C(). - n is capped at kll_constants::MAX_N (2^63 - 1), as in datasketches-java. This also keeps a merge from reaching 62 levels. - Docs: the cost of a weighted update with a small weight. - Tests: tisonkun's fixture from #582, failure at each copy of a merge and of the copy constructor, a converting constructor that rejects the order, a comparator that can't be default-constructed, the n cap; seed the "weighted update: estimation mode" section. Merge results are byte-identical to before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coverage Report for CI Build 44581048119Coverage increased (+0.01%) to 96.898%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to tisonkun's review of #582.
When both sketches held items at the same level, the merge copied other's items after this sketch's items had been moved out. A copy that threw left destroyed items counted, so the destructor destroyed them again. A weighted update reaches this once the sketch is in estimation mode, and
merge(const&)reached it before.Changes:
populate_work_arrays()copies (or moves) other's items into the work buffer first. If a copy throws, it destroys them, and this sketch is unchanged. Then this sketch's items are merged in front of them using moves only. A new helper,merge_sorted_arrays_into_gap(), replaces the version that copied fromconst T*.update()andmerge()construct an item in level zero before counting it, and reset the sorted view before any change. If a copy throws during a merge, the sketch stays valid and usable but may hold part of the other sketch. This is documented.kll_helperuses the sketch's comparator instead ofC(), so a stateful comparator is respected, and a comparator without a default constructor works.kll_constants::MAX_N(2^63 - 1), as in datasketches-java. Updates and merges that would pass it throwstd::overflow_errorand leave the sketch unchanged. This also keeps a merge from reaching the level count that madelevel_capacitythrow partway through.update(item, weight)now describes both paths. It notes that a comparator that throws leaves the sketch undefined, the same decision as for Java classic quantiles in Quantiles items: keep the sketch consistent when an update throws datasketches-java#805.Tests:
merge(const&)with overlapping levels.The four allocation tests fail on master under ASan/UBSan and pass here. Serialized merge results are byte-identical to master; I checked k = 8, 20 and 200 for float and string sketches,
const&and rvalue merges, and weighted updates. All tests pass under ASan+UBSan.🤖 Generated with Claude Code