fix(runtime): #29 isWellFormed/toWellFormed lone-surrogate detection

[proggeramlug]

Closes #29.

What was broken

String.prototype.isWellFormed() always returned true, and toWellFormed() was a no-op, because lone surrogates were silently discarded at parse time.

SWC stores string literals with lone surrogates as Wtf8Atom. Calling as_str() on a Wtf8Atom that contains lone surrogates returns None (they can't be represented as valid UTF-8). The old code at lower_patterns.rs did:

ast::Lit::Str(s) => Ok(Expr::String(s.value.as_str().unwrap_or("").to_string()))

So "\uD800" (lone high surrogate) became an empty string "" in the HIR. isWellFormed() was then checking a perfectly-valid empty string and returning true.

Approach (StringHeader flag, not full WTF-8)

A full WTF-8 migration would touch every string operation in the runtime. Instead, this PR uses a minimal flag-based approach:

1. New HIR variant Expr::WtfString(Vec<u8>) — carries raw WTF-8 bytes through the pipeline for string literals that contain lone surrogates. Regular Expr::String(String) is unchanged.

2. New runtime function js_string_from_wtf8_bytes — allocates and stores the raw WTF-8 bytes, sets STRING_FLAG_HAS_LONE_SURROGATES in the new StringHeader.flags field.

3. StringHeader gains a flags: u32 field (offset 16, struct grows 16→20 bytes). Flag propagates through js_string_concat and fused concat variants so a concatenation of a tainted + clean string is tainted.

4. isWellFormed() — O(1) flag check. Returns false iff STRING_FLAG_HAS_LONE_SURROGATES is set.

5. toWellFormed() — scans raw WTF-8 bytes. Replaces the WTF-8 lone-surrogate pattern (0xED 0xA0–0xBF 0x80–0xBF) with U+FFFD (0xEF 0xBF 0xBD). Returns a clean string via js_string_from_bytes.

Before / After

// Before (node --experimental-strip-types):
isWellFormed lone high surrogate: false
isWellFormed lone low surrogate: false
isWellFormed paired surrogates: true
toWellFormed result: [�]

// Before (Perry v0.5.190 — same file):
isWellFormed lone high surrogate: true   ← wrong
isWellFormed lone low surrogate: true    ← wrong
isWellFormed paired surrogates: true
toWellFormed result: []                  ← wrong (surrogate lost at parse time)

// After (Perry with this fix, --no-cache):
isWellFormed lone high surrogate: false  ✓
isWellFormed lone low surrogate: false   ✓
isWellFormed paired surrogates: true     ✓
toWellFormed result: [�]            ✓

Gap test delta

test_gap_string_methods: FAIL → PASS

Gap suite total: 18/28 → 25/27 (several other tests in the suite had also improved between the last recorded sweep and now; this PR is responsible only for the string_methods flip).

Test file

test-files/test_is_well_formed.ts — 20 cases covering:

• Lone high surrogate (\uD800)
• Lone low surrogate (\uDFFF)
• Properly paired surrogates (𐀀 = U+10000)
• ASCII + lone surrogate mix
• toWellFormed replacement
• isWellFormed on normal ASCII / Unicode strings
• .length of a string containing a lone surrogate (should be 1, not 0)

All 20 cases match Node --experimental-strip-types byte-for-byte.

What this does NOT fix

• Lone surrogates in identifiers, template literals, or regex character classes (separate parse paths in lower.rs / lower_types.rs)
• String.prototype.normalize() with lone surrogates
• Full WTF-8 round-trip through JSON.stringify / regex engines / encodeURIComponent (those would need a deeper WTF-8 migration)

These adjacent cases are noted in GAPS.md's "lone surrogate handling (WTF-8)" entry and are out of scope for this PR.

Struct layout note

StringHeader grows from 16 to 20 bytes. All construction sites use std::mem::size_of::<StringHeader>() so they adjust automatically. The two out-of-crate copies (perry-ui-macos/src/string_header.rs and perry-jsruntime/src/bridge.rs) are updated in this PR.

---

Generated by Claude Code