Skip to content

feat(java): add JsonReader.readRawValue for custom JSON codecs - #4230

Open
pavel-ptashyts wants to merge 4 commits into
apache:mainfrom
pavel-ptashyts:json-read-raw-value
Open

pavel-ptashyts wants to merge 4 commits into
apache:mainfrom
pavel-ptashyts:json-read-raw-value

Conversation

@pavel-ptashyts

@pavel-ptashyts pavel-ptashyts commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Why?

A custom JsonValueCodec cannot get the source text of a JSON value it reads: JsonReader.slice is protected and position() is @Internal. Codecs that accept "an object, or the same object as a JSON string" and pass the object to another parser have to copy it token by token into another writer and parse it again. Details in #4229.

What does this PR do?

  • API. public final String JsonReader.readRawValue(): reads the next complete JSON value and returns its exact source text. It is the read-side counterpart of @JsonRawValue / writeRawValue(String).
  • Behaviour. Leading whitespace is skipped and not returned; inner whitespace, quotes and escape sequences are kept verbatim; a JSON null token returns the text "null" (a codec that wants Java null checks tryReadNullToken() first). The reader stops right after the value, so the caller consumes the following comma or closing delimiter as usual.
  • Validation and limits. The span is found with the existing skipValue(), so it gets the same checks: string escapes and control characters, UTF-8 validity on the UTF-8 reader, number grammar, and maxDepth counted from the current depth. The returned String is a leaf and is not charged to the graph-memory budget; its length is bounded by input already consumed.
  • Readers. JsonReader gets a protected rawText(start, end) hook that defaults to slice. Latin-1 and UTF-16 readers use it as is. Utf8JsonReader overrides it to decode the span as UTF-8, because its slice builds a Latin-1 string and is only correct for ASCII token spans (numbers, zone ids).
  • Docs. New "Reading raw values" section in docs/json/custom-codecs.md with an object-or-string codec example; docs/json/annotations.md points to it from the JsonRawValue read note.
  • Tests.
    • ForyJsonTokenOperationTest.readRawValueKeepsSourceText: all three reader kinds on the same input (object with inner whitespace, escaped and non-ASCII string, number, true, null, empty object, newline between values), CJK and surrogate-pair text on the UTF-16 and UTF-8 readers, the root readUtf8 token API, invalid literal and empty input.
    • JsonCodecAnnotationTest.rawValueCodec (codegen and interpreted, via the existing factory): a @JsonCodec String property read with readRawValue and written with writeRawValue, round trips from Latin-1 String, UTF-16 String and UTF-8 bytes, scalar and array values, malformed values (leading zero, unterminated string, trailing comma), and maxDepth.
    • Replacing the UTF-8 override with slice makes both tests fail on non-ASCII input.

Not included, on purpose: a byte[] variant for the UTF-8 reader and a read mode for @JsonRawValue properties. The latter would change what existing models accept; both can follow if maintainers want them.

Related issues

AI Contribution Checklist

  • Substantial AI assistance was used in this PR: yes

  • If yes, I included a completed AI Contribution Checklist in this PR description and the required AI Usage Disclosure.

  • If yes, my PR description includes the required ai_review summary and screenshot evidence or equivalent persisted links of the final clean AI review results from both fresh reviewers described in AI_POLICY.md, the Fory-guided reviewer and the independent general reviewer, on the current PR diff or current HEAD after the latest code changes.

  • I performed a line-by-line self-review of all AI-assisted code changes.

  • I addressed all AI review comments and repeated the review loop until both ai reviewers reported no further actionable comments.

  • I attached screenshot evidence or equivalent persisted links of the final clean AI review results from both fresh reviewers on the current PR head in this PR body.

  • I ran adequate human verification and recorded evidence (checks run locally, pass/fail summary, and confirmation I reviewed results).

  • I added/updated tests and specs where required.

  • I validated protocol/performance impacts with evidence when applicable.

  • I verified licensing and provenance compliance.

AI Usage Disclosure
- substantial_ai_assistance: yes
- scope: design drafting, code drafting, tests, docs, PR text
- affected_files_or_subsystems: java/fory-json (JsonReader, Utf8JsonReader, tests), docs/json (custom-codecs, annotations)
- ai_review: line-by-line self-review by the contributor. Two-reviewer loop on fresh sessions every round. Round 1: the independent reviewer found a null-unsafe write in the docs example codec and a wrapping nit in annotations.md (both fixed); the Fory-guided reviewer had no comments. Round 2: the independent reviewer asked for a malformed-UTF-8 test, since `new String(..., UTF_8)` would silently replace invalid bytes if validation ever moved (added, short and word-scan cases); the Fory-guided reviewer had no comments. Round 3: both clean. Final round on the current head: both reviewers report no further actionable comments.
- ai_review_artifacts: Fory-guided reviewer: https://github.com/apache/fory/pull/4230#issuecomment-6951389321 ; independent general reviewer: https://github.com/apache/fory/pull/4230#issuecomment-6951390188 (both on head commit 501542cc, final round, no further actionable comments)
- human_verification: see "Verification" below
- performance_verification: no existing read or write path changed; the new method reuses skipValue and allocates only the returned String
- provenance_license_confirmation: all code written for this PR against the Apache Fory sources; no third-party code introduced

Verification

JDK 21.0.2, Windows 11 x86_64, base apache/main c8bb610:

  • java/: mvn -pl fory-json -am install -DskipTests, then mvn -pl fory-json test spotless:check checkstyle:check at fcdae7a: fory-json 1336 tests, 0 failures; spotless and checkstyle clean. The last commit (501542c) only adds test cases; ForyJsonTokenOperationTest and JsonCodecAnnotationTest rerun green there.
  • docs/json: prettier clean for the changed files.
  • Targeted: ForyJsonTokenOperationTest and JsonCodecAnnotationTest (both codegen modes) pass; with the Utf8JsonReader.rawText override replaced by slice, readRawValueKeepsSourceText and both rawValueCodec runs fail on non-ASCII input.

Does this PR introduce any user-facing change?

  • Does this PR introduce any public API change? JsonReader.readRawValue(). No existing behaviour changes.
  • Does this PR introduce any binary protocol compatibility change?

Benchmark

No benchmark: no existing read or write path changed.

🤖 Generated with Claude Code

pavel-ptashyts and others added 3 commits November 17, 2026 15:28
Custom codecs can read the next JSON value as its exact source text, the
read-side counterpart of @JsonRawValue and writeRawValue(String). The value
is validated by skipValue, including string escapes, UTF-8 input and the
configured maximum depth. Utf8JsonReader decodes the span as UTF-8 because
its ASCII-only slice cannot carry string content.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pavel-ptashyts

Copy link
Copy Markdown
Contributor Author

AI review artifact: Fory-guided reviewer (final round)

  • Reviewer: Codex CLI codex exec, model gpt-6-astra, reasoning effort medium, read-only sandbox, fresh session
  • Instructions: Fory-guided reviewer per AI_POLICY.md, following AGENTS.md and .agents/ci-and-pr.md
  • Reviewed: git diff apache/main...HEAD at head 501542cc (base c8bb6101)
  • Result:
No further actionable comments.

@pavel-ptashyts

Copy link
Copy Markdown
Contributor Author

AI review artifact: independent general reviewer (final round)

  • Reviewer: Claude Code CLI claude -p, model claude-opus-5-5, effort medium, read-only tools (Read Grep Glob), no MCP, fresh session
  • Instructions: independent general reviewer per AI_POLICY.md; not pointed to .agents/ci-and-pr.md or any Fory review checklist (AGENTS.md independent-review carve-out)
  • Reviewed: git diff apache/main...HEAD at head 501542cc (base c8bb6101)
  • Result:
No further actionable comments.

On JDK 8 a String is never byte-backed, so Latin1JsonReader rejects
String input there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Java] fory-json: let custom codecs read a JSON value as raw source text (JsonReader.readRawValue)

1 participant