Skip to content

build: byte-identical source compiles to a 44x-slower object depending on where the .ts file lives; the published baseline measures the fast arm #7288

Description

@proggeramlug

Summary

The same .ts file, byte-identical, compiled by the same perry binary, linking the same runtime archive, produces two different object files — and one of them runs 09_method_calls 44× slower than the other. The discriminator is where the source file lives on disk.

source at <perry-checkout>/**/09_method_calls.ts   → object 6efab0f508029e70.o →   78–110 ms
source anywhere else                               → object b0047dfba1cb3e98.o → 3417–3503 ms

benchmarks/results/public-node-bun-v1.json reports 79 ms for 09_method_calls, because benchmarks/compare.sh does cd benchmarks/suite and compiles there. A user compiling the identical file in their own project gets 3450 ms. The published baseline is measuring an arm that only exists inside the Perry checkout.

Reproduction (deterministic, 12/12 and 6/6)

P=target/release/perry
SRC=benchmarks/suite/09_method_calls.ts

# arm A — inside the checkout, any subdirectory, any filename
mkdir -p ./zzp && cp $SRC ./zzp/ && (cd ./zzp && PERRY_CACHE_DIR=/tmp/c1 ../../$P 09_method_calls.ts -o /tmp/A)
/tmp/A            # method_calls:104

# arm B — outside the checkout, same file
mkdir -p /tmp/zzp && cp $SRC /tmp/zzp/ && (cd /tmp/zzp && PERRY_CACHE_DIR=/tmp/c2 $PWD/$P 09_method_calls.ts -o /tmp/B)
/tmp/B            # method_calls:3435

Controls, all with a fresh PERRY_CACHE_DIR each time (so no stale cached object is involved):

condition result
<checkout>/benchmarks/suite/ 108 ms
<checkout>/zzp11/deep/er/ (arbitrary subdir) 104 ms
<checkout>/zzp3/renamed_bench.ts (renamed) 110 ms
/private/tmp/zzprobe/ 3426 ms
/private/tmp/a/b/c/d/e/ 3436 ms
/private/tmp/perryfake/benchmarks/suite/ (same dir names) 3501 ms
/private/tmp/perryfake2/perry/benchmarks/suite/ 3503 ms
<scratchpad>/mc_clean/ 3435 ms
/Users/amlug/projects/perry/ (checkout's parent) 3433 ms

Ruled out, each by direct test:

  • not the cwd — arm B stays slow when run from inside the checkout with an absolute path to an outside source, and arm A stays fast when run from /private/tmp with an absolute path to an in-checkout source
  • not the filename — a renamed copy inside the checkout is fast
  • not the object cache — every arm above used a freshly created PERRY_CACHE_DIR; the stored object hashes differ (6efab0f508029e70.o vs b0047dfba1cb3e98.o), so codegen genuinely differs
  • not the runtime archive — both link target/perry-auto-2f56c86280e7e60e/release/libperry_runtime.a (identical ld warning line in -vv output)
  • not auto-optimizePERRY_NO_AUTO_OPTIMIZE=1 gives 3450 ms in arm B, unchanged
  • not PERRY_WORKSPACE_ROOT — pointing it at the checkout from arm B does not help (and the workspace root is resolved from the exe path anyway, so it is already the checkout in both arms)
  • not PERRY_ALLOW_PERRY_FEATURES — no effect
  • not the installed package set — symlinking the checkout's node_modules into /private/tmp makes the wildcard expand to the same 18 packages, and the build is still slow (3451 ms)
  • not a package.json / tsconfig.json — the checkout has no perry key and no tsconfig; adding a plain package.json outside changes nothing

One further datum that is probably the thread to pull: inside the checkout, adding a package.json with "perry": {"compilePackages": []} or "perry": {"compilePackages": ["*"]} flips the build to the slow arm (3435 / 3417 ms), while no package.json at all (falling through to the checkout root's, which has no perry key, i.e. the implicit auto-default) stays fast (106 ms). So the switch is somewhere in compile/host_config.rs host-config resolution — compile_packages_explicit and friends — feeding something that reaches codegen. But it is not the expanded package set, per the symlink control above.

Full -vv diff between the two arms

The entire observable difference in the compile log is:

< Compile package wildcard: expanded to 18 installed package(s)
> Compile package wildcard: expanded to 2 installed package(s)
< perry-codegen: emitted 188906 bytes of LLVM IR for '09_method_calls.ts'
> perry-codegen: emitted 197288 bytes of LLVM IR for '09_method_calls.ts'

Same module count (Found 1 module(s): 1 native, 0 JavaScript), same clang line, same target, same archives.

What the two objects actually differ by

--trace llvm from both locations, 1147 lines of diff. The fast arm lowers this.value to the typed class-field path:

%r70 = load i64, ptr @perry_class_keys__09_method_calls_ts__Counter
%r72 = load volatile i8, ptr @PERRY_CLASS_FIELD_INLINE_GUARD_DISABLED
br i1 %r78, label %class_field_inline.deref.9, label %class_field_inline.guardcall.10

The slow arm never emits class_field_inline.* at all and instead routes every access through generic polymorphic inline caches (@perry_ic_1/2/5/7, put.dynic.*, __ic_decl_5), i.e. the js_typed_feedback_class_field_{get,set}_guard → TLS-hashmap path described in #5094. #5094 quotes method_calls at 3300 ms; that is exactly the arm reproduced here, still live in 0.5.1279.

Blast radius

Only class-field access in a hot loop diverges. Checked side by side:

benchmark in-checkout outside
09_method_calls 78 3435
16_matrix_multiply 631 642
11_prime_sieve 107 108
15_mandelbrot 22 22
bench_object_property 149 141
07_object_create 3 3
12_binary_trees 4 4
bench_gc_pressure 37 30

So the other published numbers are trustworthy; 09_method_calls is not.

Why this is more than a benchmarking embarrassment

  1. A user-visible 44× cliff with no diagnosable cause. Nothing in the compile output tells the user which arm they got. Moving a file, or adding a perry key to package.json, silently changes generated code by 44×.
  2. It makes every measurement in the repo conditional. Any A/B run from a scratch directory — the normal way ad-hoc .ts probes are measured — silently lands on a different codegen path than CI and than the published baseline. That is the "gate runs but its subject never did" failure mode from CLAUDE.md, applied to benchmarking.
  3. It hides perf(GC): make per-object layout O(1)-loadable — kill per-operation thread-local layout tracking (umbrella: method_calls/array-downgrade/object-property) #5094. perf(GC): make per-object layout O(1)-loadable — kill per-operation thread-local layout tracking (umbrella: method_calls/array-downgrade/object-property) #5094's 290× is presented as historical; it is not. It is one directory move away.

Suggested next steps

  1. Instrument the decision: log which class-field lowering was selected and why (--opt-report, cf. --opt-report: candidate-generation pre-filters for Ptr<Shape> / Ptr<NumArray> record nothing, so "0 candidates" cannot be read #7112 / feat(cli): --opt-report — surface which values could not be statically typed, why, and whether the developer can fix it #6952), so the two arms are distinguishable from the compile output.
  2. Bisect compile/host_config.rs host-config resolution against the codegen predicate that selects class_field_inline — the compilePackages-shaped controls above narrow it a long way.
  3. Make the published-baseline harness assert its arm (e.g. nm/IR check that class_field_inline is present), so a regression to the slow path shows up as a failure rather than as a number.

Measured on Apple M1 Max, macOS 26.5, perry 0.5.1279 @ defa4d601, release, auto-optimize on. Numbers are min of 5 self-reported ms; the spread within each arm is under 3%, and the in-checkout arm reproduces public-node-bun-v1.json (78 vs 79).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugConfirmed defect or regression

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions