Skip to content

density_sketch: generate serialization snapshots under -DGENERATE=true - #588

Merged
leerho merged 1 commit into
masterfrom
density-snapshots
Nov 20, 2026
Merged

leerho merged 1 commit into
masterfrom
density-snapshots

Conversation

@leerho

@leerho leerho commented Nov 19, 2026

Copy link
Copy Markdown
Member

Closes #584.

Adds density/test/density_sketch_serialize_for_java.cpp, built only with -DGENERATE=true like the other sketches' generators. It writes density_float_n{N}_cpp.sk and density_double_n{N}_cpp.sk for N = 0, 1, 10, 100, 1000 and 10000.

The inputs are the same as in the datasketches-rust generator (datagen/src/density.rs): k = 10, dim = 3, and the point {x, sqrt(x), -x} for x = 0 .. n-1. So each C++ snapshot has a Rust counterpart with the same name.

Checked locally:

  • The exact-mode images (n = 0, 1, 10) are byte-identical to the Rust snapshots in datasketches-tck, for float and double.
  • The estimation-mode images (n >= 100) differ from Rust's, as expected: compaction shuffles with std::shuffle, whose draws depend on the standard library.
  • C++ deserializes all 24 images, its own and Rust's, from bytes and from a stream, with the right n, k and dim.
  • The density tests pass under ASan and UBSan.

Follow-up in datasketches-tck once this merges: mark the C++ density_ snapshots with n >= 100 as unstable, as it does for Rust's, and re-pin C++.

🤖 Generated with Claude Code

Adds density_sketch_serialize_for_java.cpp, built only with GENERATE like
the other sketches' generators. It writes density_float_n*_cpp.sk and
density_double_n*_cpp.sk for n = 0, 1, 10, 100, 1000 and 10000, from the
same inputs as the datasketches-rust generator (k = 10, dim = 3, the point
{x, sqrt(x), -x} for x = 0 .. n-1), so each C++ snapshot has a Rust
counterpart with the same name.

Closes #584

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leerho
leerho requested a review from tisonkun November 19, 2026 20:09
@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 44605601942

Coverage decreased (-0.01%) to 96.877%

Details

  • Coverage decreased (-0.01%) from the base build.
  • Patch coverage: No coverable lines changed in this PR.
  • 2 coverage regressions across 1 file.

Uncovered Changes

No uncovered changes found.

Coverage Regressions

2 previously-covered lines in 1 file lost coverage.

File Lines Losing Coverage Coverage
common/include/serde.hpp 2 98.02%

Coverage Stats

Coverage Status
Relevant Lines: 17417
Covered Lines: 16873
Line Coverage: 96.88%
Coverage Strength: 1878941.14 hits per line

💛 - Coveralls

@tisonkun tisonkun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified that Rust can deserialize all 12 C++ density sketch snapshots and reserialize each one to exactly the original bytes.

@leerho
leerho merged commit e9a6340 into master Nov 20, 2026
35 checks passed
@leerho
leerho deleted the density-snapshots branch November 20, 2026 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

density_sketch: generate serialization snapshots under -DGENERATE=true

3 participants