Repository navigation
feat(java): opt-in lenient reading of off-spec JSON value shapes - #4235
Open
pavel-ptashyts wants to merge 15 commits into
Open
pavel-ptashyts wants to merge 15 commits into
pavel-ptashyts wants to merge 15 commits into
Conversation
Add @JsonLenient (class, field, setter, creator parameter) and ForyJsonBuilder.lenientRead(...) to accept common off-spec input shapes per property: a single value as a one-element array or collection, numbers and booleans as String text, truncated floats for integral targets, whitespace-padded quoted scalars, and empty strings as null. Lenient properties get a read-only coercion codec in front of their Fory-owned codec when metadata is built; strict properties and their generated readers are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reject an empty string decoded to null for a non-null occurrence and a null single element for a non-null array, limit lenient quote padding to spaces, and reject @JsonLenient on a property that is never read. Add tests for element-only leniency, single values into primitive and byte arrays, class settings on creator parameters, and invalid placements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…leniency Rebuild lenient array elements only when the array uses the default representation of its component, and read collection single values with the collection's own element type information, so module carriers such as unsigned arrays keep their semantics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…er leniency Skip lenient element and single-value handling for arrays and collections produced by a registered codec or codec factory; they own their element semantics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…iency Apply the registered-codec exclusion before any lenient dispatch so registered maps and other factory codecs keep their reader, and drop JsonRawValue from the excluded annotations since it only affects writing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…end lenient tests Document getter placement, which declarations are rejected, and that explicit-null handling does not apply to empty strings; cover boolean text, separate generated codecs per instance, creator codegen, and content null handling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Range-check truncated integers in JsonReader with the target's overflow message and token position, read zero with an out-of-range exponent as zero, and document that padded non-finite literals stay strict. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Property-list creators, static factories, and setter parameters now merge a parameter-level @JsonLenient like other property annotations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Limit the integral digits a truncated BigInteger may expand to by its token length (at least the long range), truncate tiny numbers with out-of-range exponents to zero, treat space-only strings as empty for trimmed numeric containers, and test element leniency through generated readers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Route integral tokens that may not fit a long through the decimal range check so FLOAT_AS_INT reports the target's overflow message, and document that EMPTY_STRING_AS_NULL skips JsonSubTypes bases and dynamic Object targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Do not re-parse whole numbers beyond the long range only to report a target-specific overflow message; the integer parser's own error is sufficient. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…part Without an exponent, parse only the integer part of a lenient truncated number after checking its digit count, document that conflicting member declarations are rejected, and test unaffected targets and one-sided padding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Report unboxed value properties separately when @JsonLenient is declared on them, document that other methods are ignored, and test non-inheritance, nested objects, getter declarations, and JsonValue types. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le-token parsing Decide the truncated integer's digit count from the mantissa digits and the clamped exponent before any allocation, and build only the surviving digits, so long mantissas with exponents no longer parse the whole token. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pavel-ptashyts
requested review from
PragmaTwice,
chaokunyang and
theweipeng
as code owners
November 17, 2026 18:37
Contributor
Author
|
AI review artifact: Fory-guided reviewer (final round)
|
Contributor
Author
|
AI review artifact: independent general reviewer (final round)
|
JsonConfig retains the builder's lenient feature set, so the enum must be build-time initialized like the other configuration enums. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
AI review artifact: Fory-guided reviewer (final round)
|
Contributor
Author
|
AI review artifact: independent general reviewer (final round)
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why?
Moving a JSON read path from Jackson to Fory JSON stops at input that is slightly off-spec but that
Jackson accepts: a lone string where a list is declared, a number where a string is declared,
300.0or" 300 "for anint,""for an absent number or object. Third-party input (OpenRTBbid requests in our case) contains all of these, and today each one needs a user codec on every
affected property, which also turns off the generated fast path for that property.
Today the request model of our ad server carries a trimming/coercing
@JsonCodecon everyStringproperty just to accept numbers and booleans as text, a collection codec whose only purpose is to
accept a lone string as a one-element list, and a Jackson fallback that re-parses the whole document
for the remaining shapes. The same
ForyJsoninstance also reads DSP responses that must staystrict, and we do not use Mixins, so the switch has to be selectable per class and per property; a
mapper-wide flag alone would not help.
Alternatives considered: mapper-wide flags only (one instance often reads both untrusted and strict
documents); a single
lenient = trueboolean (the features differ in risk,FLOAT_AS_INTloses dataand
EMPTY_STRING_AS_NULLchanges meaning, while the "all" default keeps the one-liner); attributeson
@JsonProperty(no class-level default for models with hundreds of properties); user codecs (whatwe do today); and making some coercions unconditional like #3967 (would change every strict reader).
Happy to change the shape, for example separate annotations per feature or a different name.
What does this PR do?
@JsonLenienton a class, field, getter, setter or@JsonCreatorparameter, withindependent features
SINGLE_VALUE_AS_ARRAY,SCALAR_AS_STRING,FLOAT_AS_INT,TRIM_QUOTED_SCALARandEMPTY_STRING_AS_NULL(all of them when no argument is given), andForyJsonBuilder.lenientRead(Feature...)as the default for properties without a declaration.Default: no feature, so reading is unchanged. A property declaration replaces the class setting,
which replaces the builder setting;
@JsonLenient({})makes one property strict. Like@JsonInclude, the class annotation is not inherited, and an exact-target Mixin can supply it(registered in the runtime and annotation-processor Mixin lists).
string elements of a property-owned array or collection, only where Fory owns the
representation. Properties with
@JsonCodec,@JsonFormator@JsonByteArray, and types with aregistered codec, codec factory or
@JsonValue, keep their representation. A propertydeclaration that cannot apply, or one on a never-read, Any or unwrapped property, fails when
metadata is built. Root values, map values and
Optionalcontents are not affected. Writing isunaffected. Exact rules are in
docs/json/annotations.md#jsonlenient.JsonFieldInfo/JsonCreatorFieldInfopass their effective feature mask toJsonTypeResolver.withLenientRead, next towithContentNullRead. It returns the occurrenceunchanged when nothing applies; otherwise it builds an occurrence of kind
OBJECTaround aread-only
LenientReadCodec(the mechanism@JsonFormatalready uses), and for built-in arraysand collections it rebuilds only that occurrence with lenient scalar elements, keeping its
content null handling. The codec peeks one token and delegates every native token to the wrapped
occurrence's current reader slot, so generated child capabilities stay in use. Token-level
helpers (
tryReadEmptyString,beginPaddedQuotedScalar/finishPaddedQuotedScalar,readTruncatedLong,readTruncatedBigInteger) live inJsonReaderand are reached only fromlenient codecs.
digit positions and a clamped exponent before any allocation, and builds only the surviving
digits (for
BigInteger, at most the token length with the long range as the minimum), so longmantissas and huge exponents cannot drive allocation or work. Padding accepts spaces
only, never escapes or raw control characters. Empty strings decoded to
nullare stillrejected for non-null occurrences.
unchanged. The generated-codec key includes the builder mask and each read property's mask.
JsonLenientTestruns with codegen on and off and reads every case from a Latin-1string, a UTF-16 string, UTF-8 bytes and a stream. It covers every feature and target type,
element-only leniency through generated readers, empty and space-only strings, native shapes
reading the same as strict, truncation limits (overflow, huge and tiny exponents,
BigIntegerexpansion), malformed padding, property/class/builder precedence, getters, setter and creator
parameters (named and property-list creators, static factories), Mixins, registered and module
representations, non-null occurrences, content null handling, and invalid declarations.
Related issues
Related: #3967 (quoted scalars, kept as the native path), #4095 (read-side null handling),
#4166 (
@JsonDuplicateProperties).AI Contribution Checklist
yesyes, I included a completed AI Contribution Checklist in this PR description and the requiredAI Usage Disclosure.yes, my PR description includes the requiredai_reviewsummary and screenshot evidence or equivalent persisted links of the final clean AI review results from both fresh reviewers described inAI_POLICY.md, the Fory-guided reviewer and the independent general reviewer, on the current PR diff or current HEAD after the latest code changes.Does this PR introduce any user-facing change?
Yes: a new opt-in annotation and builder option. Default behavior is unchanged.
Benchmark
JMH 1.37, JDK 25.0.2, Windows, one fork,
-Xmx512m, average time; a small OpenRTB-likeImpmodel (11 properties, nested object, list, map). Baseline is
apache/mainc8bb6101; strict caseswere run in adjacent base/current pairs. The host was shared with other heavy jobs, so the error
bars are wide; the strict code path itself is unchanged. Measured on the first commit of the
branch; later commits changed only lenient-only code.
@JsonLenientclass, well-formed UTF-8@JsonLenientclass, well-formed Latin-1@JsonLenientclass, off-spec UTF-8🤖 Generated with Claude Code