perf(codegen): stamp a pointer-free shape's typed layout into the allocation header

Four allocation benchmarks sat within a 6% band at 2.50-2.65x node, which is
the signature of one shared per-allocation cost rather than four problems.

Profiling the band (200M-allocation variants, two samples each, agreeing
within 1.5pp) found it: `js_gc_declare_typed_shape_layout` was 30% of
`churn_alloc` and `push_cls`, and it spent that re-deriving per OBJECT a fact
that is a property of the SHAPE. #7510's memo had already reduced the map
round-trip to a direct-mapped probe; what remained was the probe itself, a
type-table lookup, a field-count compare, and the cross-crate call.

For a shape whose pointer mask is statically EMPTY the answer is a constant:
`GC_LAYOUT_POINTER_FREE | GC_OBJ_TYPED_LAYOUT_INTACT`. The inline-bump `new`
path already writes a packed `GcHeader` constant that carries the state half,
so the intact bit is folded into the same store and the call disappears. What
survives is the one half that depends on the recycled ADDRESS rather than the
shape - clearing a previous tenant's per-object record - now a one-argument
`js_gc_forget_object_layout` behind a `PERRY_PER_OBJECT_LAYOUTS_ANY` test whose
`0` state proves every thread's per-object tables empty.

Two smaller levers in the same band:

* `js_ctor_return_override` was called per construction to answer a question
  that is `undefined` for every constructor without an explicit `return` -
  8% of `churn_alloc`. `JSValue::is_undefined` is `bits == TAG_UNDEFINED`, so
  one 64-bit compare decides it inline and the runtime call stays on the cold
  arm, where derived-constructor TypeErrors and object returns still need it.

* A `new` in a function the hot-loop-callee pre-pass admitted is a `new` in a
  loop one frame out, so it takes the inline bump too. `cycles.ts`'s
  `makeCycle` is the shape: 5 statements, hence `alwaysinline` and hence never
  `inlinehint`, so the existing gate read the one flag it could not have.

Measured on the quiet M1 mini, best-of-5, outputs byte-identical to node with
exit 0 verified for all 27 programs:

| bench | before | after |
|---|--:|--:|
| churn | 0.4217 | 0.2900 |
| churn_alloc | 0.3720 | 0.2409 |
| push_cls | 0.3665 | 0.2368 |

`churn_alloc` goes 18.6 -> 12.0 ns per allocation (node is 7.1 on the same
shape). `gc-handoff/bench/alloc_declare_{pf,ptr}.ts` isolate it: identical
programs differing only in whether the second field's declared type makes the
pointer mask non-empty.

Soundness: the collector's view is bit-identical. `heap_payload_slot_selection`
skips a `GC_LAYOUT_POINTER_FREE` payload without consulting any map, and the
pre-existing path also reached `POINTER_FREE` for an empty pointer mask. A
later pointer store still downgrades - with no descriptor to classify against,
`layout_note_slot` falls through to its generic pointer-mask branch, which
mints a per-object mask and flips the state to `SIDE_MASK`, needing no
descriptor at all. A pointer-BEARING shape keeps the full runtime declare,
because its `SIDE_MASK` state means the tracer reads a mask and that call is
what installs it.

Author: Ralph Küpper

crates/perry-codegen/src/codegen/function.rs

@@ -426,6 +426,12 @@ pub(super) fn compile_function(
     // functions (the hint would be redundant) and async/generator forms.
     // Try-containing functions are ordinary inline candidates since #7302
     // (invoke-EH removed the setjmp-era noinline requirement).
+    // #7815: record the raw admission, before the `inline_hint` window narrows
+    // it. `lower_call/new_alloc.rs` uses this to decide the inline-bump
+    // allocation, which wants "is this code hot" and not "may LLVM's inline
+    // threshold move" — an `alwaysinline` callee is excluded from the latter
+    // and is the hottest possible case for the former.
+    lf.hot_loop_callee = cross_module.hot_loop_callees.contains(&f.id);
     if !lf.force_inline
         && inline_hot_small_enabled()
         && (INLINE_HOT_SMALL_MIN..=inline_hot_small_size_cap()).contains(&f.body.len())

crates/perry-codegen/src/function.rs

@@ -36,6 +36,18 @@ pub struct LlFunction {
     /// inline-hot-small heuristic in `codegen/function.rs`. `alwaysinline`
     /// already implies the hint, so the two are never emitted together.
     pub inline_hint: bool,
+    /// #7815: `collectors::collect_hot_loop_callees` admitted this function —
+    /// it has at least one direct call site inside a LOOP and at most
+    /// `inline_hot_small_max_call_sites` call sites in the whole module.
+    ///
+    /// Not the same question as [`Self::inline_hint`], which is that set
+    /// INTERSECTED with a body-length window and with "not already
+    /// `alwaysinline`". A ≤8-statement function is `alwaysinline` and therefore
+    /// never hinted, yet it is exactly the shape whose body ends up executing
+    /// once per loop iteration — `cycles.ts`'s `makeCycle`. Sites that want
+    /// "is this code hot?" rather than "should LLVM's threshold move?" read
+    /// this.
+    pub hot_loop_callee: bool,
     /// Invoke-EH (#7302): this function contains landing pads (Itanium) or
     /// funclet pads (SEH), so its `define` line must carry
     /// `personality ptr @<name>` — `perry_eh_personality` on Mach-O/ELF,
@@ -215,6 +227,7 @@ impl LlFunction {
             linkage: String::new(),
             force_inline: false,
             inline_hint: false,
+            hot_loop_callee: false,
             personality: None,
             blocks: Vec::new(),
             block_counter: 0,

crates/perry-codegen/src/gc_call_effects.rs

@@ -83,6 +83,9 @@ pub(crate) fn classify_direct_callee(name: &str) -> GcCallEffect {
         | "js_gc_note_slot_layout_aware"
         | "js_gc_init_typed_shape_layout"
         | "js_gc_declare_typed_shape_layout"
+        // #7815: `layout_forget_object` behind a null check — two thread-local
+        // side-table removals, no allocation and no re-entry.
+        | "js_gc_forget_object_layout"
         // `typed_feedback.rs`: counters/registries only. This intentionally
         // does not include feedback wrappers that perform the actual object
         // get/set operation.

crates/perry-codegen/src/lower_call/mod.rs

@@ -71,6 +71,8 @@ mod scalar_method;
 /// module header for why the default build cannot fault on them.
 #[cfg(test)]
 mod timer_rooting_tests;
+#[cfg(test)]
+mod typed_shape_bake_tests;
 /// #7510: which of the two typed-shape layout entry points a `new` site emits,
 /// and where. Split out of `new.rs` to keep it under the 2000-line cap.
 mod typed_shape_init;

crates/perry-codegen/src/lower_call/new.rs

@@ -500,7 +500,8 @@ fn lower_new_impl_inner<'a>(
 
     // #7615 slice 8: the field-count computation and the three-arm instance
     // allocation moved verbatim to `new_alloc.rs` (see its header for why).
-    let obj_handle = super::new_alloc::emit_instance_alloc(ctx, class_name, class);
+    let alloc = super::new_alloc::emit_instance_alloc(ctx, class_name, class);
+    let obj_handle = alloc.handle;
     // #7154: root the instance for the duration of the constructor body.
     //
     // Until now the instance existed ONLY as an SSA register while that body
@@ -532,7 +533,7 @@ fn lower_new_impl_inner<'a>(
     //
     // Before the instance root's push, so the handle this names is the one the
     // allocator returned: nothing between here and there can collect.
-    emit_typed_shape_layout_declare(ctx, class_name, &obj_handle);
+    emit_typed_shape_layout_declare(ctx, class_name, &obj_handle, alloc.typed_layout_baked);
     let instance = {
         let protected = construction_runs_user_code(ctx, class_name);
         Instance {
@@ -669,16 +670,8 @@ fn lower_new_impl_inner<'a>(
                 || class.extends_name.is_some()
                 || class.native_extends.is_some()
                 || class.extends_expr.is_some();
-            let is_derived_lit = if is_derived { "1" } else { "0" };
-            let final_box = ctx.block().call(
-                DOUBLE,
-                "js_ctor_return_override",
-                &[
-                    (DOUBLE, &obj_box),
-                    (DOUBLE, &ctor_ret),
-                    (crate::types::I32, is_derived_lit),
-                ],
-            );
+            let final_box =
+                super::new_helpers::emit_ctor_return_override(ctx, &obj_box, &ctor_ret, is_derived);
             return Ok(final_box);
         }
         if let Some(save) = &saved_new_target {
@@ -1560,16 +1553,7 @@ fn lower_new_impl_inner<'a>(
         }
         ctx.current_block = after_idx;
         let raw = ctx.block().load(DOUBLE, &ret.result_slot);
-        let is_derived = if ret.is_derived { "1" } else { "0" };
-        ctx.block().call(
-            DOUBLE,
-            "js_ctor_return_override",
-            &[
-                (DOUBLE, &obj_box),
-                (DOUBLE, &raw),
-                (crate::types::I32, is_derived),
-            ],
-        )
+        super::new_helpers::emit_ctor_return_override(ctx, &obj_box, &raw, ret.is_derived)
     } else {
         obj_box
     };

crates/perry-codegen/src/lower_call/new_alloc.rs

@@ -42,9 +42,42 @@ use crate::types::{I32, I64, I8, PTR};
 /// scan-outward-past-switch-frames logic uses. A `new` inside a bare `switch`
 /// is therefore correctly treated as not-in-a-loop.
 fn new_site_is_in_loop(ctx: &FnCtx<'_>) -> bool {
-    ctx.loop_targets
+    if ctx
+        .loop_targets
         .iter()
         .any(|(continue_label, _, _)| !continue_label.is_empty())
+    {
+        return true;
+    }
+    // #7815: a `new` in a function the hot-loop-callee pre-pass admitted is a
+    // `new` in a loop, one frame out.
+    //
+    // The gate below this comment is about SPEED-vs-SIZE, and
+    // `collect_hot_loop_callees` answers exactly the question the loop test
+    // does — is this site hot enough to be worth ~268 bytes — with the
+    // anti-bloat backstop already attached: it admits only a function that
+    // (a) has a direct call site inside a loop and (b) has at most
+    // `inline_hot_small_max_call_sites` (4) direct call sites in the whole
+    // module. So the added code is bounded by 4 × (news in the function),
+    // which is the same order the loop arm already accepts.
+    //
+    // Deliberately NOT `func.inline_hint`: that is this set intersected with a
+    // 9..=20-statement window and with "not already `alwaysinline`", and the
+    // functions this needs most fall out of BOTH. `makeCycle` is 5 statements,
+    // so it is `alwaysinline` and never hinted — while being the single
+    // hottest function in the program.
+    //
+    // `cycles.ts` is the shape that needs it: `makeCycle` is called 10M times
+    // from `main`'s loop and allocates two `Cell`s, but its own body has no
+    // loop, so both allocations took the outlined
+    // `js_object_alloc_class_inline_keys` — 22% of the program's samples, plus
+    // a further 5% in `arena_alloc`'s inline-state sync, for work the inline
+    // bump does in eight stores.
+    //
+    // Reading `func.hot_loop_callee` here is well-ordered: `codegen/function.rs`
+    // sets it from `cross_module.hot_loop_callees` before the entry block is
+    // created and before any expression is lowered.
+    ctx.func.hot_loop_callee
 }
 
 /// Emit the instance allocation for `new <class_name>(...)` and return the raw
@@ -62,7 +95,37 @@ fn new_site_is_in_loop(ctx: &FnCtx<'_>) -> bool {
 /// emission, which is the `RootedGroup::adopt_emitted` push that roots it for
 /// the constructor body; nothing between the allocator call and that push can
 /// collect.
-pub(super) fn emit_instance_alloc(ctx: &mut FnCtx<'_>, class_name: &str, class: &Class) -> String {
+/// What [`emit_instance_alloc`] produced: the instance's user pointer, plus
+/// whether the allocation already stamped this class's canonical typed-shape
+/// layout into the object's `GcHeader` constant (#7815).
+pub(super) struct InstanceAlloc {
+    pub(super) handle: String,
+    /// `true` ⟹ the header already reads `GC_LAYOUT_POINTER_FREE |
+    /// GC_OBJ_TYPED_LAYOUT_INTACT`, so the construction site owes the runtime
+    /// only the address-dependent half of `js_gc_declare_typed_shape_layout`
+    /// (clearing a recycled address's stale per-object record).
+    pub(super) typed_layout_baked: bool,
+}
+
+pub(super) fn emit_instance_alloc(
+    ctx: &mut FnCtx<'_>,
+    class_name: &str,
+    class: &Class,
+) -> InstanceAlloc {
+    let mut typed_layout_baked = false;
+    let handle = emit_instance_alloc_inner(ctx, class_name, class, &mut typed_layout_baked);
+    InstanceAlloc {
+        handle,
+        typed_layout_baked,
+    }
+}
+
+fn emit_instance_alloc_inner(
+    ctx: &mut FnCtx<'_>,
+    class_name: &str,
+    class: &Class,
+    typed_layout_baked: &mut bool,
+) -> String {
     // Compute total field count including inherited parent fields.
     // The runtime allocates at least 8 inline slots regardless, so this
     // mostly matters for shapes >8 fields.
@@ -296,8 +359,41 @@ pub(super) fn emit_instance_alloc(ctx: &mut FnCtx<'_>, class_name: &str, class:
             // `js_gc_note_slot_layout` so the GC sees real pointer-bearing
             // slots regardless of this initial tag.
             const GC_LAYOUT_POINTER_FREE: u64 = 0x4000;
+            /// `GC_OBJ_TYPED_LAYOUT_INTACT` — the bit
+            /// `class_field_inline_guard` requires before it will read or write
+            /// a raw-f64 slot directly. Runtime-side name:
+            /// `gc::layout::GC_OBJ_TYPED_LAYOUT_INTACT`.
+            const GC_OBJ_TYPED_LAYOUT_INTACT: u64 = 0x1000;
             const OBJECT_TYPE_REGULAR: u64 = 1;
 
+            // #7815: when this class's canonical layout is declarable at
+            // allocation AND its pointer mask is statically empty, the state
+            // this header already carries (`GC_LAYOUT_POINTER_FREE`) is the
+            // FINAL one, and the only thing `js_gc_declare_typed_shape_layout`
+            // would add per instance is the intact bit. Stamping it into the
+            // same constant store removes the call: on `churn_alloc` /
+            // `push_cls` that call was ~30% of the program, almost all of it
+            // re-deriving per object a fact that is a property of the SHAPE
+            // (see `gc::shape_install`'s module docs — the memo already reduced
+            // the map round-trip to a direct-mapped probe, and what is left is
+            // that probe, the type-table lookup, and the call itself).
+            //
+            // Requires `field_count == slot_count`: that mismatch is the one
+            // case `init_typed_shape_layout` answers by DOWNGRADING
+            // (`layout_set_typed_unknown`), and a constant cannot express "it
+            // depends". Computed here, before `ctx.block()` takes its mutable
+            // borrow.
+            *typed_layout_baked = super::typed_shape_init::layout_pointer_free_at_allocation(
+                ctx,
+                class_name,
+                field_count,
+            );
+            let typed_intact_bits = if *typed_layout_baked {
+                GC_OBJ_TYPED_LAYOUT_INTACT
+            } else {
+                0
+            };
+
             let alloc_field_count = std::cmp::max(field_count as u64, MIN_FIELD_SLOTS);
             let payload_size = object_header_size + alloc_field_count * FIELD_SLOT_SIZE;
             // Round the whole allocation up to FIELD_SLOT_SIZE (8). The inline
@@ -410,7 +506,7 @@ pub(super) fn emit_instance_alloc(ctx: &mut FnCtx<'_>, class_name: &str, class:
             //   bits 32..63  = size (u32)
             let gc_packed: u64 = GC_TYPE_OBJECT
                 | (GC_FLAG_ARENA << 8)
-                | (GC_LAYOUT_POINTER_FREE << 16)
+                | ((GC_LAYOUT_POINTER_FREE | typed_intact_bits) << 16)
                 | ((total_size as u64) << 32);
             // GC_STORE_AUDIT(INIT): inline headers initialize freshly allocated unpublished object storage.
             blk.store(I64, &gc_packed.to_string(), &raw);

crates/perry-codegen/src/lower_call/new_helpers.rs

@@ -585,3 +585,63 @@ pub(crate) fn ctor_chain_uses_new_target(ctx: &FnCtx<'_>, class: &Class) -> bool
     }
     false
 }
+
+/// ECMAScript constructor return-override, with the `undefined` case decided
+/// **inline** (#7815).
+///
+/// Every `new` site has to apply the spec's return-override rule: a
+/// constructor that returns an Object yields that object, `undefined` yields
+/// the implicit `this`, and any other primitive yields `this` for a base
+/// constructor or throws for a derived one. `js_ctor_return_override` decides
+/// all three.
+///
+/// The overwhelmingly common case is the one it can answer from a single
+/// 64-bit compare: a constructor with no `return <expr>` at all completes as
+/// `undefined`, and the runtime's answer for `undefined` is *exactly*
+/// `this_val` — `JSValue::is_undefined` is `bits == TAG_UNDEFINED`, and
+/// `constructor_return_overrides_this(undefined)` is `false` at its first
+/// `is_pointer()` test. Emitting that compare here turns a cross-crate call
+/// into a never-taken branch: measured at 8.2% of `churn_alloc` and `push_cls`,
+/// where the synthesized object-literal constructor's only `ret` is the
+/// `TAG_UNDEFINED` constant.
+///
+/// The slow arm is byte-for-byte the previous emission, so derived-constructor
+/// `TypeError`s, `return new Promise(…)`, arguments objects and arrays all keep
+/// the runtime's answer. Skipping the call cannot lose a relocation either:
+/// `js_ctor_return_override` is in `root_reload`'s no-reload set, so no live
+/// value's address depends on having made it.
+pub(super) fn emit_ctor_return_override(
+    ctx: &mut FnCtx<'_>,
+    obj_box: &str,
+    ctor_ret: &str,
+    is_derived: bool,
+) -> String {
+    let override_idx = ctx.new_block("ctor_ret.override");
+    let merge_idx = ctx.new_block("ctor_ret.merge");
+    let override_label = ctx.block_label(override_idx);
+    let merge_label = ctx.block_label(merge_idx);
+    let this_pred_label = ctx.block_label(ctx.current_block);
+    {
+        let blk = ctx.block();
+        let bits = blk.bitcast_double_to_i64(ctor_ret);
+        let is_undef = blk.icmp_eq(crate::types::I64, &bits, crate::nanbox::TAG_UNDEFINED_I64);
+        blk.cond_br(&is_undef, &merge_label, &override_label);
+    }
+    ctx.current_block = override_idx;
+    let is_derived_lit = if is_derived { "1" } else { "0" };
+    let overridden = ctx.block().call(
+        DOUBLE,
+        "js_ctor_return_override",
+        &[(DOUBLE, obj_box), (DOUBLE, ctor_ret), (I32, is_derived_lit)],
+    );
+    let override_pred_label = ctx.block_label(ctx.current_block);
+    ctx.block().br(&merge_label);
+    ctx.current_block = merge_idx;
+    ctx.block().phi(
+        DOUBLE,
+        &[
+            (obj_box, &this_pred_label),
+            (&overridden, &override_pred_label),
+        ],
+    )
+}