Skip to content

gc: the relocating minor produces a deterministic wrong answer in the SHIPPED default on an ordinary event-loop workload #7055

Description

@proggeramlug

Found while measuring PERRY_GC_MOVING_LOOP_POLLS (measurement writeup: ISSUE_LINK). Reporting separately because it is not a poll bug and it is more urgent than the measurement.

Summary

An ordinary event-loop-driven TypeScript program produces a deterministic wrong answer in the shipped default configuration. The same binary produces the Node-correct answer when the GC is configured so that no relocating minor runs.

The discriminator is exactly copied_objects > 0. Not minor_cycles > 0, not the loop poll, not the nursery cap.

Reproducer

w5_srv_scale.ts — an event-loop request pump; each "request" builds 1,500 small objects in one synchronous stretch, folds them into a checksum, then yields via setImmediate. Request count from argv[2]. Full source at the bottom.

$ perry w5_srv_scale.ts -o w5
$ ~/node-v26.5.0-darwin-arm64/bin/node --expose-gc --experimental-strip-types w5_srv_scale.ts 150
checksum:-341887099

$ ./w5 150                                          # shipped default
checksum:-90240474                                  # WRONG

$ PERRY_GC_MOVING_LOOP_POLLS=0 ./w5 150             # no relocating minor
checksum:-341887099                                 # correct

The discriminator is relocation, not the flag

Same binary throughout; only the runtime environment differs. checksum is deterministic across repeats in every arm (5 repeats each, plus a warmup).

arm env minors @150 / @600 copied objects checksum @150 checksum @600
shipped 8 / 56 915,961 / 11,661,882 -90240474 1986912948
cap raised PERRY_GC_SCAVENGE_NURSERY_MB=128 0 / 3 0 / 0 -341887099 1735266323
polls off PERRY_GC_MOVING_LOOP_POLLS=0 0 / 1 0 / 0 -341887099 1735266323
polls off + 16 MB cap PERRY_GC_MOVING_LOOP_POLLS=0 PERRY_GC_SCAVENGE=1 8 / 60 1,045,097 / 11,351,683 -90240474 1986912948

Read the cap raised row at 600 requests: it runs 3 minor cycles and still gets the right answer, because none of them copied anything. Every arm that relocated is wrong; every arm that did not relocate is right, regardless of how it got there.

In the shipped arm the error is a fixed constant

requests shipped correct difference
150 -90240474 -341887099 251,646,625
300 -1346563643 -1598210268 251,646,625
600 1986912948 1735266323 251,646,625
1200 1973861391 1722214766 251,646,625

251646625 = 0x0EFFD2A1. Identical at every workload length, so in this configuration it is one corruption of fixed magnitude, occurring once, early — not per-request drift and not an accumulating error. The checksum is a running |0 sum over per-request folds, so a constant total offset means exactly one request's fold read exactly one wrong value.

The constant is not universal across configurations: the polls off + 16 MB cap arm reproduces the same 251,646,625 offset at 150, 300 and 600 requests but produces a third value (-1005199879) at 1200. So the magnitude is stable for a given collector configuration over a wide range, not a property of the defect itself.

How far it narrows

Partially, and not to something small:

variant result
w5 as written (async + setImmediate + JSON.stringify + argv bound) wrong at 150/600, correct at 10/50
same, PERRY_GEN_GC=0 (full mark-sweep) correct — confirms the generational/moving path
same arithmetic, const REQUESTS = 600 instead of argv correct (this is w1_srv_pump)
same arithmetic, no async, no JSON.stringify, argv bound correct at 10/50/150
as above, objects reduced to two numeric fields correct at 10/150

So it needs enough allocation to trigger a relocating minor and something the synchronous reduction does not have. The async/setImmediate structure is implicated, which is consistent with the known-weak-area note that the async-to-generator transform boxes every body local into a single shared mutable cell typed Any. I did not get below the ~60-line w5 form.

Independent corroboration

A near-identical program with the request count as a compile-time constant (const REQUESTS = 600) rather than read from argv is byte-identical to Node in the shipped configuration, and its checksum is 1735266323 — the same value the non-relocating arms produce for w5 at 600. So the correct answer is confirmed twice, by Node and by a second Perry configuration, and the difference between the two programs is only where the loop bound comes from.

Why this looks like the known class, and why it matters more than it did

This is the shape of #6951 / #6972 / #6981 / #6982: a reference held in a register or a raw unboxed slot across a relocating collection, stale afterwards.

What is new is where it reproduces. #6981 measured that class under PERRY_CONSERVATIVE_STACK_SCAN=off and concluded that the forced ManualGcScanGuard::force_full_scan() at gc_check_trigger is what keeps it out of production:

The only reason precise roots are not already the shipped behaviour on the automatic paths is that gc_check_trigger forces ManualGcScanGuard::force_full_scan() on both arms.

On the deferred/moving path that mask is not present. PERRY_GC_TRACE=1 on the shipped arm of this workload shows root_sources.native_stack_fallback.decision = "skip_disabled" on 62 of 63 cycles — the single scanning cycle is the explicit gc() at the end. So the shipped moving minor is running with precise roots and no conservative scan, which is exactly the configuration #6981 shows breaks 14 of 20 representation-corpus files.

It also needs no special arm at all — just the default configuration on a plain program — which is a concrete instance of the gap #6993 describes.

Environment

  • Commit e279b2d54, release build, cargo build --release -p perry -p perry-runtime-static -p perry-stdlib-static
  • perry aeac6a6e562cf6b1…, libperry_runtime.a 58a83af57503a84e…, libperry_stdlib.a 645fb9c483d46bb5…
  • macOS 26.5.1, arm64, 8-core M1
  • Oracle: Node v26.5.0 (.node-version pin)
  • PERRY_NO_AUTO_OPTIMIZE not set — this is the production path

Source

declare function gc(): void;

const REQUESTS = process.argv[2] ? parseInt(process.argv[2], 10) : 600;
const ROWS = 1500;

interface Row { id: number; name: string; tags: string[]; score: number; }

let checksum = 0;

function handle(req: number): number {
  const rows: Row[] = [];
  for (let i = 0; i < ROWS; i++) {
    rows.push({
      id: req * ROWS + i,
      name: "row-" + i + "-" + req,
      tags: ["a" + (i & 15), "b" + (i & 7), "c" + (i % 5)],
      score: (i * 7919 + req * 104729) % 1000,
    });
  }
  let acc = 0;
  for (let i = 0; i < ROWS; i++) {
    const r = rows[i];
    acc = (acc + r.id + r.score + r.name.length + r.tags[0].length) | 0;
  }
  const body = JSON.stringify(rows.slice(0, 24));
  return (acc + body.length) | 0;
}

async function main(): Promise<void> {
  for (let r = 0; r < REQUESTS; r++) {
    checksum = (checksum + handle(r)) | 0;
    await new Promise<void>((resolve) => { setImmediate(resolve); });
  }
  gc();
  const mu = process.memoryUsage();
  console.log("probe:w5_srv_scale");
  console.log("requests:" + REQUESTS);
  console.log("checksum:" + checksum);
  console.error("#gcmetric heap_used_bytes=" + mu.heapUsed);
  console.error("#gcmetric heap_total_bytes=" + mu.heapTotal);
  console.error("#gcmetric rss_bytes=" + mu.rss);
}

main();

Related: #6981, #6993, #6951, #6972, #6982, #6950, #6977.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions