Skip to content

fix(schema): preserve const/enum in transformJSONSchema - #1143

Open
edenbuilds wants to merge 2 commits into
anthropics:mainfrom
edenbuilds:fix/preserve-const-enum-json-schema
Open

edenbuilds wants to merge 2 commits into
anthropics:mainfrom
edenbuilds:fix/preserve-const-enum-json-schema

Conversation

@edenbuilds

@edenbuilds edenbuilds commented Aug 6, 2026 •

Copy link
Copy Markdown

Summary

Related to #1116

Test plan

  • yarn jest tests/helpers/transform-json-schema.test.ts tests/helpers/beta/zod.test.ts
  • betaZodOutputFormat(z.discriminatedUnion(...)) keeps const on each branch

Dumping leftover const/enum into description strips discriminants from
Zod discriminated unions / literal branches, so the model often emits
nested union values as JSON strings and tool.parse() fails (anthropics#1116).

Keep const and enum on the strict schema before the leftover-keys dump.
@edenbuilds
edenbuilds requested a review from a team as a code owner August 6, 2026 23:33
@percymcn

percymcn commented Aug 9, 2026

Copy link
Copy Markdown

This fix is right, but I think it lands on a different path than the issue it closes, and that's worth settling before the regression test gets pinned to the wrong helper.

betaZodTool never calls transformJSONSchema, so const is already preserved on the tools path. I checked helpers/beta/zod.js on 0.100.1 (the version in #1116's env block) and on 0.116.0 — betaZodTool is byte-identical in both and only calls z.toJSONSchema(inputSchema, { reused: 'ref' }). Running #1116's exact discriminated-union repro:

// betaZodTool(...).input_schema        -> discriminant intact
"kind": { "type": "string", "const": "set_title" }

// betaZodOutputFormat(...).schema      -> discriminant demoted
"kind": { "type": "string", "description": "{const: \"set_title\"}" }

So #1116's stated mechanism ("betaZodTool produces an input_schema where the discriminant is stripped out") doesn't reproduce. Their downstream symptom (tool.parse() → expected object, received string) may well be real, but on the tools path it has some other cause, and merging this won't move it. Two concrete suggestions: pin the new regression test to betaZodOutputFormat/jsonSchemaOutputFormat rather than betaZodTool, and ask #1116 to re-run against current betaZodTool before this is closed as fixing it.

Second, const/enum isn't a special case — it's the default for everything. _transformJSONSchema rebuilds each node from an allowlist and then does:

if (Object.keys(jsonSchema).length > 0) {
  strictSchema['description'] = ... + '{' + entries.map(...) + '}';
}

Every unrecognised key lands in description. What actually survives per node is $ref, $defs, type, anyOf/oneOf/allOf, description, title; on objects properties/required/additionalProperties; on strings format but only the 10 in SUPPORTED_STRING_FORMATS; on arrays items, and minItems only when it is exactly 0 or 1. So a plain z.string().min(3).max(80) or z.array(...).min(2).max(5) silently stops being enforced too. Measured:

"title": { "type": "string", "description": "{minLength: 3, maxLength: 80}" }
"tags":  { "type": "array", "items": {...}, "minItems": 1, "description": "{maxItems: 5}" }

Patching const/enum alone fixes the discriminator case but leaves the rest, and the failure mode is uniquely nasty because nothing throws or warns — the constraint just quietly becomes a sentence. If the intent is broader, an allowlist that keeps validation keywords the API accepts (rather than a per-keyword patch) would close the whole family at once.

Third, two cases in here are worse than demotion, and neither is touched by a const/enum patch:

Root $ref discards the entire schema. _transformJSONSchema returns immediately on $ref, before $defs is copied. zod-to-json-schema's named output goes in as {$ref: "#/definitions/Ticket", definitions: {...}} and comes out as exactly:

{"$ref":"#/definitions/Ticket"}

Dangling pointer, whole schema gone, nothing thrown. (A non-root definitions bag survives only as a JSON.stringify inside the root description, with its $refs still dangling — the transformer only knows $defs.)

Tuples fail two different ways. Array-form items — and prefixItems next to items: false — recurse into a node with no type and throw JSON schema must have a type defined if anyOf/oneOf/allOf are not used, which never mentions tuples, so it reads as a bug in the user's schema. Meanwhile a bare prefixItems, which is exactly what z.toJSONSchema(z.tuple([z.number(), z.number()])) emits, is demoted, leaving {"type":"array"} — no item schema and no length at all. The same "no type" throw also fires for any bare enum without a sibling type.

Lastly, #1117 was the same fix and was closed unmerged on 2026-07-31 with no comments. Might be worth finding out why before this one waits on a review — if there was a design objection, it probably applies here too.

(Context on how I got this: I maintain a small tool that normalises schemas per provider, and I derive its rules by running them through each vendor's own client rather than off the docs — the numbers above are all from transform-json-schema.js in 0.116.0. Happy to share the probe script if useful: https://github.com/percymcn/llm-json-schema)

betaZodTool does not call transformJSONSchema; cover betaZodOutputFormat and
clarify comments so the regression matches the real call path.
@edenbuilds

Copy link
Copy Markdown
Author

@percymcn thanks — this was exactly the review this PR needed.

You’re right that betaZodTool never calls transformJSONSchema. I re-checked helpers/beta/zod.ts: tools go straight from z.toJSONSchema → input_schema, so #1116’s stated tools-path mechanism doesn’t reproduce on current main. The transformer is used by betaZodOutputFormat / jsonSchemaOutputFormat (and the non-beta helpers), which is where const/enum demotion still bites.

Updated in f272bf1:

On the broader allowlist / root $ref / tuples: agree those are real and nastier. Happy to do a follow-up that keeps API-accepted validation keywords instead of one-off const/enum, but I don’t want to expand this PR into a general schema rewriter without maintainer signal (and without knowing why #1117 closed unmerged with no comments). If you have context on #1117’s close, I’m all ears.

Probe script link is useful — thanks for sharing.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants