fix(codegen): make FFI provenance part of the object cache entry (#6439)

[proggeramlug]

Problem

Effect's api.ts (the effect-compiled-experiment repro, #6439) linked from a cold object cache and failed from a warm one with Undefined symbols: _js_ws_connect_start — same sources, same flags, same compiler. Whether a program linked depended on whether node_modules/.cache/perry happened to be populated.

Root cause

The ext_registry (#835/#846) records which external-provider FFI symbols codegen emits; the driver folds that into needs_stdlib + native_module_imports, which is what brings perry-ext-* wrapper archives onto the link line. But the registry is populated as a side effect of codegen: record_ffi_call fires from LlBlock::call. An object-cache hit returns the cached .o and skips compile_module entirely — so nothing records the provider, the well-known flip never happens, and the wrapper silently drops off the link line.

The trigger doesn't need import "ws" anywhere: @effect/platform's Socket.ts lowers js_ws_connect_start directly (compiled but never imported), which is exactly the case the registry exists to cover — and exactly the case import-driven feature detection can't see.

Fix

FFI provenance is a property of the compiled object, so it now lives with it:

• ext_registry: a thread-local capture scoped around compile_module attributes emissions to one module (perry-codegen uses no rayon, so a module's codegen runs start-to-finish on its worker thread). replay_ffi_symbols re-records a manifest as if codegen had just run. Manifests store symbol names, not resolved owners, so routing always uses today's table rather than a stale decision baked into a cache entry.
• object_cache: each <key>.o gains a sorted <key>.ffi manifest. lookup_path_with_ffi reports a hit only when both halves are present — a manifest-less .o was written by a pre-fix perry, its provenance is unknowable, and guessing "no providers" is precisely this bug. Reporting a miss recompiles that module once and the entry self-heals; no cache wipe, no user action.
• run_pipeline: hit → replay the manifest; miss → capture and store it (manifest first, so a .o is never visible without one).

Also registers the 16 ws symbols codegen emits (js_ws_connect_start and friends) against perry-ext-ws — previously only the 3 *_client_i64 entries were routed.

Not ws-specific

The manifests written for the Effect build also record js_readable_stream_new_from_source_object — the streams routing from #835/#846 carried the same latent warm-cache bug.

Validation

• Effect api.ts: links and runs byte-identical to node on both a cold and a warm cache (311/311 objects reused), with no PERRY_FORCE_WELL_KNOWN override. web.ts links too (its remaining runtime failure is GetIterator on a class declaration throws 'is not iterable' — INT32 ClassRef rejected by js_get_iterator's primitive guard (blocks Effect's yield* SomeTag) #6454, unrelated).
• cargo test -p perry object_cache: 46 passed (3 new: manifest round-trip, empty-manifest hit, legacy-object miss).
• cargo test -p perry-codegen --lib ext_registry: 8 passed (3 new: capture collects/sorts/dedups, capture reset between modules, replay routes like live codegen).
• cargo fmt --all -- --check clean.

Fixes #6439.

https://claude.ai/code/session_01R4phoN4f7eFbSQTkuB2oQP

Summary by CodeRabbit

• Bug Fixes

  • Improved WebSocket (“ws”) linking detection, including when WebSocket usage occurs without an explicit import.
  • Fixed cached builds to link external functionality consistently with fresh builds by replaying per-module external symbol provenance.
  • Prevented incomplete/legacy cache entries from being treated as valid by requiring the associated provenance manifest.

• Performance

  • Enhanced build caching reliability by preserving and restoring per-module external symbol information across cache hits.

[coderabbitai]

📝 Walkthrough

Walkthrough

The FFI registry now covers additional WebSocket symbols and captures per-module registry usage. Object-cache entries persist FFI manifests, while the compile pipeline replays manifests on cache hits and stores them after cache misses.

Changes

FFI provenance and cache integration

Layer / File(s)	Summary
Registry coverage and symbol capture crates/perry-codegen/src/ext_registry.rs	Adds WebSocket provider mappings, thread-local module capture, replay APIs, and deterministic capture tests.
FFI manifest cache contract crates/perry/src/commands/compile/object_cache.rs, crates/perry/src/commands/compile/object_cache/object_cache_tests.rs	Adds atomic manifest writes, strict FFI-aware lookups, cache behavior coverage, key invalidation tests, and cache-directory resolution tests.
Compile and cache replay integration crates/perry/src/commands/compile/run_pipeline.rs	Captures and stores FFI symbols for newly compiled modules and replays them for cached modules.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Compiler
  participant ObjectCache
  participant FFIRegistry
  participant Linker
  Compiler->>ObjectCache: Lookup object and FFI manifest
  ObjectCache-->>Compiler: Cached object and symbols
  Compiler->>FFIRegistry: Replay symbols
  FFIRegistry-->>Compiler: Restore providers
  Compiler->>Linker: Link with selected providers

Loading

Possibly related PRs

• PerryTS/perry#5185: Updates the same FFI registry mechanism for routing emitted symbols to well-known providers.

Suggested reviewers: andrewtdiz, thehypnoo

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and accurately summarizes the main change: making FFI provenance part of object-cache entries.
Description check	✅ Passed	The PR description covers the problem, fix, linked issue, and validation, though it uses custom headings instead of the template labels.
Linked Issues check	✅ Passed	The code changes address #6439 by capturing and replaying FFI symbols across cache hits and preserving ws provenance for linking.
Out of Scope Changes check	✅ Passed	All changes are directly tied to object-cache FFI provenance and ws symbol routing; no clear unrelated additions stand out.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/6439-object-cache-ffi-provenance

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

crates/perry/src/commands/compile/object_cache.rs (1)

1018-1036: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Do not store the object after its manifest write fails.

This silently creates an incomplete cache entry: run_pipeline.rs proceeds to store the .o, but every FFI-aware lookup must reject it and recompile. Return Result/bool, record the failure, and gate the object store on success; add a manifest-write-failure test.

Proposed direction

-pub fn store_ffi_manifest(&self, key: u64, symbols: &[&str]) {
+pub fn store_ffi_manifest(&self, key: u64, symbols: &[&str]) -> bool {
     let Some(path) = self.ffi_manifest_path_for(key) else {
-        return;
+        return false;
     };
     // ...
-    if fs::write(&tmp_path, body)
+    let stored = fs::write(&tmp_path, body)
         .and_then(|_| fs::rename(&tmp_path, &path))
-        .is_err()
-    {
+        .is_ok();
+    if !stored {
         let _ = fs::remove_file(&tmp_path);
+        self.store_errors.fetch_add(1, Ordering::Relaxed);
     }
+    stored
 }

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/perry/src/commands/compile/object_cache.rs` around lines 1018 - 1036,
Update ObjectCache::store_ffi_manifest to return a success indicator or error
instead of silently ignoring write/rename failures, while preserving cleanup of
temporary files. In the run_pipeline.rs flow that stores the object, capture
this result and skip the .o cache write when manifest persistence fails. Add a
test covering manifest-write failure and confirming the object is not stored.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@crates/perry/src/commands/compile/object_cache.rs`:
- Around line 1018-1036: Update ObjectCache::store_ffi_manifest to return a
success indicator or error instead of silently ignoring write/rename failures,
while preserving cleanup of temporary files. In the run_pipeline.rs flow that
stores the object, capture this result and skip the .o cache write when manifest
persistence fails. Add a test covering manifest-write failure and confirming the
object is not stored.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b36b4146-4033-420b-a50c-68510a285ca5

📥 Commits

Reviewing files that changed from the base of the PR and between 28d421a and 7f50bde.

📒 Files selected for processing (2)

• crates/perry/src/commands/compile/object_cache.rs
• crates/perry/src/commands/compile/object_cache/object_cache_tests.rs