GC/dylib: host-boundary full collection corrupts a later imported-handler JSON Buffer response

[proggeramlug]

Summary

In a provider-hosted Perry application dylib, a synchronous full collection at
a clean host boundary can invalidate later execution of an imported handler.
The handler should return a PCH2-framed JSON Buffer; after the collection it
instead returns a valid PCH2 frame whose body is:

{"type":"Buffer","data":[0]}

The expected body is:

{"runtime":"perry","iterations":100,"checksum":3726872593}

This is independent of Next.js. It blocks bounded per-application memory for a
server that loads the Perry runtime and stdlib once and executes application
dylibs on demand.

Tested baseline

• Perry 0.5.1510, commit 0da668c95150d78ba5aa2b8eff7b78c04cc381c1
• Linux aarch64, kernel 6.8.0-117-generic
• runtime provider SHA-256: b0eb547b7be17dabe818ca23b866140d8f9491c7d6440357908554600fa9359e
• stdlib provider SHA-256: 0cb243fd69859081ac6273f3353ecf195f535e5f40d9747ae967228b84e3532e
• application package SHA-256: fd5e2026749377ad236e9426ce2b29a609f98e79238f322800038b61ccd2d769
• fixture source SHA-256: 243cc0cfffff65d8fa831a2cc0f6c8029b3f5423c9dee2c200cbbe3d8c78b9ef

The application is compiled with --no-codegen --no-auto-optimize --march generic --output-type dylib. The application does not contain runtime or
stdlib implementation code. Both providers are loaded process-wide before the
application, followed by js_gc_init() and perry_module_init() on the
application's dedicated thread.

Complete fixture generator

Save as make-perry-provider-gc-fixture.sh and run it. It creates the entire
application; no framework or application design is required.

make-perry-provider-gc-fixture.sh

#!/usr/bin/env bash
set -euo pipefail

fixture="${1:-perry-provider-gc-fixture}"
mkdir -p "$fixture/handlers"

cat > "$fixture/perch.toml" <<'TOML'
name = "worker-benchmark"
version = "0.1.0"

[hosts]
domains = ["benchmark.local"]

[[handlers]]
file = "handlers/main.ts"
path = "/api/benchmark"
method = "GET"
TOML

cat > "$fixture/handlers/main.ts" <<'TS'
export function handle(_frame: Buffer): Buffer {
  const body = Buffer.from(JSON.stringify({
    runtime: "perry",
    iterations: 100,
    checksum: 3726872593,
  }));
  const output = Buffer.alloc(5 + 2 + 4 + 4 + body.length);
  output[0] = 0x50;
  output[1] = 0x43;
  output[2] = 0x48;
  output[3] = 0x32;
  output[4] = 2;
  let offset = 5;
  output.writeUInt16BE(200, offset);
  offset += 2;
  output.writeUInt32BE(0, offset);
  offset += 4;
  output.writeUInt32BE(body.length, offset);
  offset += 4;
  body.copy(output, offset);
  return output;
}
TS

printf 'created %s\n' "$fixture"

The generated application entry is deliberately a second module, matching the
provider-hosted shape:

import { handle as perchHttpHandler } from "./handlers/main";
export function perchHttpEntry(frame: Buffer): any {
  return perchHttpHandler(frame);
}

Deterministic host sequence

1. Load the exact runtime provider globally.
2. Load the exact stdlib provider globally.
3. Load the app-only dylib eagerly and bind its imported entry alias.
4. On one dedicated application thread, call js_gc_init() and then
   perry_module_init().
5. Invoke perchHttpEntry with a valid PCH2 request Buffer, copy and validate
   the returned frame, and repeat. Fifty callers may queue work, but application
   execution remains serial on this thread.
6. Every 256 completed invocations, sample js_arena_stats. Once live bytes
   have grown at least 1 MiB above the previous post-collection baseline, call
   js_gc_memory_pressure(2) after the prior application call has completely
   returned and its response bytes have been copied.
7. Continue validating every response byte for byte.

On the attached minimal fixture the first full collection occurred at
invocation 2,048:

before_live=1065408
before_reserved=1310720
js_gc_memory_pressure(2)=2
after_live=131184
after_reserved=655360
reclaimed_live_bytes=934224

The following workload then failed with:

expected runtime perry, received undefined;
response={"type":"Buffer","data":[0]}

Controls

• With host-boundary collection disabled, all 20,000 responses pass, but the
  dead request/response buffers remain in the old generation and process PSS
  grows materially.
• A tiny handler that returns Buffer.from("ok") survives repeated boundary
  collections. The failure requires the more realistic imported handler with
  fresh object serialization and response framing.
• The collection reports return code 2, so this is not a deferred pressure
  request.
• The pressure call happens only after generated code has returned. No Perry
  frame is active above the call and no application value is held by the host.

Why this is a Perry correctness issue

js_gc_memory_pressure(2) documents this no-active-frame host boundary as a
precise synchronous full-collection point. A successful return must preserve
module bindings, runtime roots, object/JSON behavior, and subsequent handler
results. The host cannot safely infer or pin Perry's internal object graph.

Using a very small collection interval can hide this particular manifestation,
but it is not a correctness fix and adds substantial CPU cost. Disabling
collection preserves responses but leaves old-generation Buffer churn
unbounded until Perry's own much larger trigger.

Acceptance criteria

• Add a Perry-owned integration test that compiles the exact two-module fixture
  as an app-only dylib and loads it against separate runtime and stdlib provider
  images.
• Run at least 20,000 validated invocations and at least ten successful
  js_gc_memory_pressure(2) collections at clean host boundaries.
• Require every post-collection status, frame length, and body byte to match the
  pre-collection oracle.
• Include serial and queued/concurrent caller variants; Perry execution itself
  remains thread-affine and serial.
• Add a retained module-level Buffer variant and prove that both retained
  values and temporary request/response buffers are classified correctly.
• Require dead temporary buffers to be reclaimed and the latter-half live-arena
  slope to remain flat.
• Gate Linux aarch64 and x86_64. Run the same test on macOS as a portability
  control.
• Do not solve this by disabling full collection, retaining every temporary, or
  adding a host/framework-specific response path.