Skip to content

codegen: object emission is nondeterministic on Linux — the LLVM temp module name embeds pid + nanotime #7131

Description

@proggeramlug

Split out of #7128 finding E, because it is not a representation-selection
problem and it blocks a general technique.

What happens

Compiling the same file twice with identical flags produces different object
bytes on aarch64 Linux
and identical bytes on macOS.

Measured (#7128): all 26 benchmarks/repsel_census workloads were
nondeterministic on a Raspberry Pi 5 (aarch64, Linux), 0 of 26 on an M1 Mac
mini. On suite_01_startup exactly 10 bytes differ.

Cause

crates/perry-codegen/src/linker.rs:294-306 names the temporary IR file after
the process:

let pid = std::process::id();
let counter = TEMP_NONCE_COUNTER.fetch_add(1, Ordering::Relaxed);
let wall_nonce = SystemTime::now().duration_since(UNIX_EPOCH).map(|d| d.as_nanos());
let ll_path  = tmp_dir.join(format!("perry_llvm_{pid}_{wall_nonce}_{counter}.ll"));

clang records that path as the translation unit's file name, and on ELF it
lands in the emitted object. Mach-O keeps it in the debug map rather than the
.o, which is why macOS looks clean — the same non-determinism is present, it
just is not in the bytes being compared.

The uniqueness requirement is real (#509: two rayon workers racing the same
microsecond overwrote each other's .ll), so the fix is not "drop the nonce" —
it is to make the name a function of the content rather than of the clock,
e.g. perry_llvm_<hash(ll_text)>_<counter>.ll, or to pass clang
-fdebug-compilation-dir / -frandom-seed and normalise the recorded name.

Why it matters beyond reproducible builds

Object-hash A/B is the cheapest and least deniable instrument this project has
for "did this change what ships" — #7113, #7119, #7121, #7128 all rest on it,
and #7128's whole "which representation moved which workload" table is an
object-hash table. That instrument does not work on Linux today. Every
object-level claim in #7128 had to be taken on the Mac mini, which in turn
meant no instruction-retired counter (macOS has no unprivileged one), so
"emission changed" and "it got slower" could not be measured on the same host.

It also defeats:

Repro

perry compile benchmarks/suite/01_startup.ts -o /tmp/a.o --no-link --no-cache
perry compile benchmarks/suite/01_startup.ts -o /tmp/b.o --no-link --no-cache
# on aarch64 Linux: the two "Wrote object file:" paths differ in 10 bytes

(Compare the paths the compiler prints, not the -o arguments — --no-link
does not honour -o.)

Acceptance

Compiling every census workload twice with identical flags produces
byte-identical objects on Linux as well as macOS, and
census-knob-isolation --require-emission passes on the Pi.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions