exp: in-process LLVM backend — stop shipping IR as text (Phase 0: feasible, byte-identical objects)

[proggeramlug]

Branch: exp/llvm-inprocess · Write-up: docs/llvm-inprocess-experiment.md on that branch.

Perry ships LLVM IR as text to a user-supplied clang -c — a runtime, unpinned LLVM dependency (1.4 GB transient IR on the Claude Code bundle, #4880 whole-module -Os demotion, Apple clang 21 vs LLVM 22 parse skew, errors as line numbers in gigabyte files). This experiment builds the module pipeline in-process via the LLVM C API, exactly like the exp/stackmap-viability pattern: isolated branch, opt-in, default path untouched.

Phase 0 answer (feasibility: CONFIRMED)

• Bindings: inkwell 0.9.0 + llvm-sys 221 for the few unwrapped calls (LLVMSetGC, LLVMParseCommandLineOptions, LLVMGetVersion). Nothing Perry's IR needs was inexpressible: NaN-box constants keep exact payload bits through f64::from_bits → const_float (verified end-to-end by running the compiled artifact), inline asm with exact constraint strings, module-level asm, appending @llvm.used, and gc "statepoint-example" all construct, verify, and emit.
• LLVM version: 22 (llvm22-1 / Homebrew llvm 22.1.4). Picking 22 removes the version-skew problem — no second toolchain parses our output anymore.
• Discovery: LLVM_SYS_221_PREFIX (or llvm-config on PATH) at build time.
• Contributor cost: none by default. The llvm-inprocess cargo feature is off; the default build links no LLVM and is byte-for-byte unchanged. Enabling it needs LLVM 22 dev libs and adds ~171 MB to a static-linked perry-dev binary.

Parity evidence

• On the same IR + flags, the in-process pipeline emits an object byte-identical to clang -c (Homebrew clang 22).
• 77-test gap-corpus A/B through an unmodified perry (shim as PERRY_LLVM_CLANG; both caches already key on it): 76/77 outputs byte-identical; the 1 diff is a pre-existing perry-ext-http SIGABRT whose stderr embeds the OS thread id (differs run-to-run on the same arm).
• 26-test slice through the integrated PERRY_LLVM_INPROCESS=1 backend (one binary, flag-only A/B, liveness asserted per compile): 25/26 identical, same pre-existing panic family for the remainder.
• .ll → parse → print round-trip diff classes are all mechanical (alignment made explicit, attr-group renumbering, float formatting, headers) — the normalization catalogue for a future construction-parity harness.

What's on the branch

• experiments/llvm-inprocess-spike/ — standalone spike: builder-API demo, clang-argv shim, corpus A/B script.
• crates/perry-codegen/src/inprocess.rs behind the llvm-inprocess feature: compile_ll_to_object gets an in-process implementation (memory-buffer parse → verify → default<O_n> → object-to-memory; zero disk transit). It interprets the same build_clang_compile_plan argv as the clang path, so backend decisions can't drift independently; unknown flags are a hard error. PERRY_LLVM_INPROCESS participates in build-cache and object-cache keys; a featureless build fails loudly under the flag; first use prints a liveness witness line.

Honest negatives / open work

• Compile-time win not demonstrated at small scale (245 KB–421 KB modules, loaded host): subprocess spawn isn't the bottleneck; -O3 is. The claimed wins live at bundle scale (parse + disk transit), in version control, and in what the seam unlocks (per-function opt vs Compile-time blowup on wide object literals (2100 keys ≈ 2 min, 3000 keys > 7 min) #4880, DWARF via DIBuilder, in-process RS4GC for gc: RewriteStatepointsForGC over managed-pointer SSA — the measured-only road to native-root file-size parity #7174). Those need a quiet box, ≥25 GB disk, and the gap suite green under the flag before any number is quoted.
• The brief's instruction-level seam ("lowering ↔ IR construction") is not where this codebase's seam is: the emitter is ~340k lines of string-oriented code, and compile_ll_to_object is the boundary the code itself documents as "the seam". Recommendation in the write-up: land the transport seam opt-in, profile string-build cost through it at scale, and only then decide the construction rewrite.
• -fno-math-errno has no C-API equivalent; measured byte-identical on macOS (Darwin defaults errno-free) — must be re-measured on Linux.
• cl::opt state is process-global; Windows/COFF and cross-targets untested; llvm-sys/inkwell version pins are ours to maintain.

Next steps

1. Gap suite (full) under PERRY_LLVM_INPROCESS=1, macOS + Linux.
2. Claude Code bundle compile through the flag on a quiet box: fill the measurement table (wall/user, peak RSS, transient disk, binary size/sections, runtime + RSS vs pinned Node).
3. Coordinate with gc: RewriteStatepointsForGC over managed-pointer SSA — the measured-only road to native-root file-size parity #7174 before scheduling RewriteStatepointsForGC in-process — that branch owns the statepoint work.
4. Decide per-function opt levels + DIBuilder scope from the measured profile.

[proggeramlug]

Phase 2 landed (opt-in): native IR construction — module-scale text is gone under PERRY_LLVM_INPROCESS=native

Owner direction after Phase 0: the transport isn't the lever — making the generated .ll optional is. That now exists on the branch, defaults untouched.

What =native does: every function body is built through the LLVM C API. The only text that still exists per module is the few-KB skeleton (globals, declares incl. synthesized ones for defines, attribute groups) plus transient per-function scaffolding that is dropped immediately after construction. The 1.4 GB-scale module concatenation, the disk transit, and the full-grammar LLVM parse no longer exist on this path. --trace llvm / PERRY_LLVM_KEEP_IR remain the debug view.

Architecture (differs from the original brief's builder-trait seam, deliberately): two audits showed 96.4% of instruction emission (6,097 call sites) already funnels through ~40 semantic methods on LlBlock, but LlFunction::to_ir() performs load-bearing post-render passes (entry-alloca hoists, boundary splices, per-ret shadow-frame rewrites, setjmp volatile promotion). So the port point is the finalized function: a bounded dialect reader (perry-codegen/src/dialect.rs, census-driven, fails loudly on unknown forms) consumes each finalized function and constructs it natively — those transforms keep exactly one implementation. Next step to kill the per-line scaffolding too: typed LlInst variants inside LlBlock, migrated method-by-method with the diff harness as gate; the per-module instructions= counter is the ratchet.

Corpus-measured traps encoded in the reader (each cost a real failure first):

• Non-phi cross-block forward references are real (the megamorphic idispatch.tower shape) → placeholder-RAUW, LLParser-style. 73% of phi incomings are forward references.
• Under opaque pointers, a direct call's function type is the callsite's, not the declare's (js_native_call_value declared (double,i64,i64), called with (double,ptr,i64)) → call types derive from callsites.
• The C-API builder constant-folds at construction (zext i1 false, select i1 false, …, constant GEPs), so pre-opt print comparison is structurally impossible → =diff's verdict is emitted object bytes (pre-opt IR dumps via PERRY_LLVM_DIFF_DIR remain the localization artifact).
• Skeleton globals reference defined wrapper functions (extern-closure descriptors); external is an explicit linkage keyword on init_body defines; atomic load ordering trails the pointer operand.

Results:

• Corpus gate (unit tests): every function of both .ll corpora (9,110+ instructions) constructs natively and passes the verifier.
• spike.ts: text-parse arm and native-construction arm emit byte-identical objects; the native binary matches the pinned Node oracle.
• batch.ts: object bytes differ by 336 B of tighter native code — verified at disassembly level to be regalloc/layout divergence downstream of pre-folded constants, same symbols; all pre-opt diff lines are the folding class, none are construction bugs.
• Gap-corpus slice under =native (one binary, flag-only A/B, per-compile liveness asserted): 26/26 byte-identical outputs (one initially failed compile exposed the linkage + skeleton-declare bugs, fixed; one harness false-negative was a warm object cache suppressing the liveness line — re-verified cold).
• cargo test -p perry-codegen --features llvm-inprocess: 528 passed. Default (featureless) build unaffected.

Still open before any default flip: full 466-test gap suite under =native (macOS + Linux), bundle-scale compile with RSS measurement (the folding-divergence class should also be re-checked at that scale), the typed-LlInst migration to remove per-function transient text, per-function opt levels / DWARF / RS4GC (#7174 coordination) that the in-memory module now makes reachable.

[proggeramlug]

Full gap-suite gate under PERRY_LLVM_INPROCESS=native: closed (macOS)

Full 466-test A/B (text arm vs native-construction arm, same binary, flag-only difference, per-compile liveness asserted):

• 459/466 byte-identical stdout+stderr+exit status; every native compile proven live; 0 compile failures, 0 backend-attributable divergences.
• 7 residual DIFFs, all pre-existing runtime noise verified reproducible between two runs of the same arm: 5× perry-ext-http server.rs:911 SIGABRT and 1× tokio listener panic (stderr embeds the OS thread id), 1× console.time() wall-clock jitter (0.032 vs 0.033 ms).

The full sweep earned its keep twice over the sampled slices:

1. Folded-placeholder miscompile (class/prototype family, (number).set is not a function): the forward-reference placeholder select true, undef, undef was constant-folded by the C-API builder to the uniqued constant undef — un-RAUW-able and verifier-clean, silently substituting undef for every non-phi forward reference. Also explains the earlier batch.ts 336-byte object delta previously misread as benign regalloc divergence: it was this bug deleting never-executed code. Fixed (placeholder is now a load from a scratch alloca — unfoldable, erased on resolution; use/def type mismatch is a hard error). Post-fix, both corpora emit byte-identical objects across construction paths, and the harness rule is strict: byte mismatch = bug, there is no benign class. Localization recipe that found it: =diff dumps + opt -passes=instcombine over BOTH arms + diff (representation folds vanish, semantics survive).
2. Intrinsic auto-declaration: perry's clamp lowering emits @llvm.smax.i32/smin.i32 calls with no textual declare — LLVM's .ll parser auto-declares intrinsics at first use, so the text path never noticed. The reader now mirrors that (for llvm.* names only; other undeclared callees remain loud errors).

Also landed since the last update: typed LlInst storage behind the emit choke point (Raw variant, byte-identical by the 528-test gate) and LlFunction::for_each_final_line — a single finalized-order visitor shared by to_ir and the native builder, which now streams lines directly into the C API with no per-function text materialized (exception: has_try functions, whose setjmp volatile pass needs whole-function analysis). Remaining transient text is per-line Raw formatting, removed opcode-by-opcode as typed variants land; per-module instruction counts are the ratchet.

Still open before any default consideration: Linux run of this same sweep, bundle-scale compile with RSS measurement on a quiet box, and the typed-variant migration of the ~40 LlBlock semantic methods.

[proggeramlug]

Current state + Linux bring-up (handoff)

Branch exp/llvm-inprocess @ 2c2c5c0f0 is fully pushed. Summary of where things stand (details in the two comments above):

Works, validated (macOS/arm64): PERRY_LLVM_INPROCESS=native builds all function bodies through the LLVM C API for standard single-module compiles — full 466-test gap A/B: 459 byte-identical, 0 compile failures, 0 backend divergences (7 residuals are pre-existing panic/thread-id + timer noise). Both benchmark corpora emit byte-identical objects across construction paths. =diff is the harness (object-byte verdict; PERRY_LLVM_DIFF_DIR dumps both arms).

Known gaps (honest list):

1. Codegen-unit-split (oversized) modules fall through to in-process transport (text, no clang) — the bundle-scale modules are exactly these; porting unit-split to native is the top engineering item.
2. Per-line text still exists: instructions are still format!-ed once and parsed by the dialect reader. The typed-LlInst migration (~40 LlBlock semantic methods; storage + streaming visitor already landed) removes it. has_try functions additionally materialize function text (setjmp volatile pass needs whole-function analysis).
3. Debug knobs are dead under =native: PERRY_LLVM_KEEP_IR, PERRY_SAVE_LL, --trace llvm currently no-op silently there (they sit on the text path). Small fix: print from the constructed module. Until fixed, use =diff + PERRY_LLVM_DIFF_DIR for inspection.
4. Everything below is macOS-validated only.

Linux bring-up (fresh clone is all you need — nothing lives only on the Mac):

git clone https://github.com/PerryTS/perry && cd perry
git checkout exp/llvm-inprocess
# LLVM 22 dev libs (apt.llvm.org):
wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && sudo ./llvm.sh 22
sudo apt-get install -y llvm-22-dev clang-22 zlib1g-dev libzstd-dev libtinfo-dev
export LLVM_SYS_221_PREFIX=/usr/lib/llvm-22

cargo build --profile perry-dev -p perry -p perry-runtime-static -p perry-stdlib-static \
  --features perry/llvm-inprocess

Linux test plan, in order:

1. cargo test --profile perry-dev -p perry-codegen --features llvm-inprocess --lib — 528 tests incl. the corpus gates (the corpus .ll files are now tracked; if these print "skipping", that is a bug, not a pass).
2. Single-module smoke: compile experiments/llvm-inprocess-spike/spike.ts with and without PERRY_LLVM_INPROCESS=native (set PERRY_RUNTIME_DIR=$PWD/target/perry-dev, PERRY_NO_AUTO_OPTIMIZE=1); outputs must match and the native run must print perry: in-process LLVM backend active (LLVM 22.x).
3. =diff byte-verdict on spike + benchmarks/app-patterns/kernels/batch.ts — expect [ir-diff] OK. (Two ELF traps pre-fixed on the branch: diff arms share one module identifier since ELF records it in STT_FILE; the A/B harness now uses timeout and a portable disk probe.)
4. Full sweep: experiments/llvm-inprocess-spike/batch_ab.sh /tmp/gap-linux flag 1 0 with INPROCESS_VALUE=native. Compare against the macOS triage above; anything outside the known noise families is new signal.
5. math-errno measurement (Linux-specific open question): grab a kept .ll (PERRY_LLVM_KEEP_IR=1 on a text-path compile), then
   clang-22 -c -O3 -mcpu=native x.ll -o a.o vs the same plus -fno-math-errno, and cmp a.o b.o. On macOS they are byte-identical (Darwin defaults errno-free); if they differ on glibc, the in-process pipeline needs an equivalent knob before Linux parity can be claimed.
6. Report results here either way — a Linux-specific divergence family is a real result.

Merge-to-main (opt-in) checklist beyond Linux: CI arm building --features llvm-inprocess + running the suite + one liveness-asserted =native smoke (land non-required, then promote); changelog.d fragment; reviewer callout that the LlInst/visitor refactor touches the default path's to_ir (byte-identity pinned by the 528-test suite); security-audit green with inkwell 0.9/llvm-sys 221 as optional deps.

[proggeramlug]

This is now layer 0 of the consolidated GC plan — see docs/src/internals/rfc-rooting-by-construction.md ("Where this sits in the one GC correctness plan", landed in #7292).

Its importance is larger than "stop shipping IR as text". #7108 measured statepoint/stack-map GC as viable but blocked, concluding that "the text-IR-plus-stock-clang architecture is what rules the cheapest design out". This experiment removes that constraint, and its own Phase 0 independently verified that gc "statepoint-example" constructs, verifies and emits through the C API.

⇒ Statepoints (#7108/#7174) depend on this landing first. That makes this the enabler for the layer that would eliminate an entire class of GC bugs by construction — #7280, #7271, #7252 and #7243 all become unrepresentable once the emitted code's liveness is tracked by LLVM rather than by a hand-maintained shadow stack.

Worth stating so it is weighed: the payoff is not only the IR-as-text problems (1.4 GB transient IR, #4880's whole-module -Os demotion, errors as line numbers in gigabyte files) but control of the pass pipeline and stackmap emission, which is what makes the 4.5–16.6 MB metadata cost negotiable rather than fixed.

[proggeramlug]

Linux bring-up: closed. 459/466 A/B, -fno-math-errno answered, harness made portable + resumable

Measured at 89b7dd191 on Fedora 43 / x86_64 / glibc / LLVM 22.1.8 / ELF — i.e. before the five commits the Mac landed while this ran (typed LlInst, unit-split native, RS4GC). Scope note at the bottom.

Test plan results, in the handoff's order

#	Step	Result
1	perry-codegen --features llvm-inprocess --lib	528/528, same count as macOS. Both corpus gates confirmed constructing, not skipping
2	spike.ts text vs =native	stdout/stderr/exit byte-identical; liveness line printed; matches the Node oracle
3	=diff byte verdict	[ir-diff] OK on spike.ts (31,929 B) and batch.ts (84,345 B)
4	Full 466-test gap A/B under =native	459 SAME, 7 DIFF, 0 compile failures on either arm, 466/466 live, 0 NOT_LIVE. 71 min wall
5	-fno-math-errno on glibc	No-op. Question closed — see below

The two construction paths are byte-identical on ELF/x86-64, not just Mach-O/arm64. That is the result I'd most want carried forward: the placeholder-RAUW, callsite-derived call types, and skeleton-declare machinery are not accidentally Mach-O-shaped.

The 7 residuals are not backend-attributable — proven, not asserted

Same three families as macOS, and I applied the branch's own standard rather than pattern-matching them: re-ran one arm's binary twice and diffed it against itself. All 7 differ against themselves.

• 5× perry-ext-http server.rs:911 SIGABRT (exit 134 on both arms) — stderr embeds the OS thread id
• 1× tokio listener.rs:304 panic, test_gap_net_connect_bound_value (exit 124 both arms)
• 1× console.time() wall-clock jitter — test_gap_console_methods, 0.008 vs 0.009 ms; the same binary re-run gives 0.002 vs 0.001 ms

-fno-math-errno: closed, and for a structural reason

The open question assumed the risk was "glibc defaults the other way". The real answer is that the flag has nothing to act on: it only governs whether LLVM may treat a call to a named libm function as errno-free, and Perry never emits one. Math lowers either to llvm.* intrinsics (errno-free by definition) or to js_math_* runtime helpers, whose libm behavior lives inside libperry_runtime.a and is not produced by this compile at all.

Measured, not just argued — on a 431 KB real module and on a probe exercising every Math.* entry point plus frem (10 intrinsic call sites, 6 frems, zero named libm callees): objects byte-identical with and without the flag, under both clang 22.1.8 and clang 19. The in-process backend needs no equivalent knob on glibc.

The load-bearing property is "no named libm callee in emitted IR". If the emitter ever gains one, this needs re-measuring — worth a cheap assertion if this graduates.

Fedora needed a route the handoff's recipe doesn't cover

Fedora 43 has no LLVM 22 package at any version (repos stop at llvm20-devel; system libs are 21), so apt.llvm.org has no analogue. Used the upstream release tarball (LLVM-22.1.8-Linux-X64.tar.xz, ~12 GB unpacked, ships headers + llvm-config + static libLLVM*.a + clang 22).

One trap worth knowing before someone else loses an hour: that tarball is built on Ubuntu, so its baked-in llvm-config --system-libs names the build host's static zstd by absolute path (/usr/lib/x86_64-linux-gnu/libzstd.a). llvm-sys panics on any --system-libs entry that is neither -lfoo nor an existing file, so on any non-Debian distro the build dies before it starts. Fix is a one-file shim prefix whose bin/llvm-config rewrites that to -lzstd; documented in the write-up.

Feature build cost here: perry 199 MB (vs 53 MB default), 2m49s at perry-dev.

Harness fixes (89b7dd191)

batch_ab.sh could not start on Linux at all: TIMEOUT=$(command -v "$TIMEOUT") dies under set -u (the intent was coreutils' macOS gtimeout), and WT was a hardcoded /Users path. Both fixed; the free-space probe now looks at the filesystem the work dir is actually on rather than /.

Two guards earned the hard way: a concurrent cargo build pushed this box into the OOM killer, which killed six compiles mid-link. They landed in results.txt as CFAIL_BOTH — after the fact, indistinguishable from six real pre-existing failures — and cost a full restart of a ~72-minute run. So the harness now resumes (tests already recorded are skipped) and never scores a compile that died on a signal: it waits for memory, retries once, and if it dies again records OOM_INFRA on its own summary line, a category no backend conclusion may be drawn from. Both counters read 0 on the clean run.

Also removed a vacuous green: the corpus gate's missing-file branch printed "skipping" and returned pass. The corpora have been tracked since 4f001d39e, so absence means a broken checkout — it is a hard error now. That branch was exactly what the handoff told me to rule out by hand.

Peak RSS per compile (gap-test scale)

244 MB text arm vs 273 MB native arm — the in-process context costs ~29 MB here. Says nothing about the bundle-scale RSS question, which is still open.

Scope limits

• PERRY_NO_AUTO_OPTIMIZE=1 on both arms.
• Box's node is v22.22.2 against the 26.5.1 pin. Doesn't affect a perry-vs-perry A/B, but this run makes no oracle-parity claim — the spike.ts oracle check above is a single file.
• At 89b7dd191 unit-split modules still fell through to transport, so the sweep did not exercise native construction on them. e370f5554 has since ported that path.

Suggested next on Linux

1. Re-run the 466 A/B at branch HEAD — the typed-LlInst migration and the unit-split native path are both unvalidated on ELF, and the unit-split one is the bundle-scale shape.
2. PERRY_CODEGEN_UNITS=3 =diff on Linux, mirroring e370f5554's macOS check.
3. Bundle-scale compile + RSS on a quiet box — this one has 25 GB and a browser on it, which is how the OOM happened.