x.py: `bootstrap` keeps being recompiled

[SimonSapin]

Every x.py command seems to start like this for me:

./x.py something something
Updating only changed submodules
Submodules updated in 0.01 seconds
   Compiling libc v0.2.39
   Compiling dtoa v0.4.2
   Compiling lazy_static v0.2.11
   Compiling cfg-if v0.1.2
   Compiling num-traits v0.2.1
   Compiling unicode-xid v0.1.0
   Compiling serde v1.0.29
   Compiling getopts v0.2.15
   Compiling cc v1.0.6
   Compiling itoa v0.3.4
   Compiling proc-macro2 v0.2.3
   Compiling num_cpus v1.8.0
   Compiling time v0.1.39
   Compiling filetime v0.1.15
   Compiling build_helper v0.1.0 (file:///home/simon/rust3/src/build_helper)
   Compiling quote v0.4.2
   Compiling cmake v0.1.29
   Compiling syn v0.12.14
   Compiling serde_derive_internals v0.20.0
   Compiling serde_json v1.0.10
   Compiling toml v0.4.5
   Compiling serde_derive v1.0.29
   Compiling bootstrap v0.0.0 (file:///home/simon/rust3/src/bootstrap)
    Finished dev [unoptimized] target(s) in 12.12 secs
Building stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[…]

Sometimes bootstrap and its dependencies are all rebuilt twice within the same x.py command!

12 seconds is not a lot compared to building two stages of rustc, but if it’s something that’s not too hard to fix it’s not negligible either.

[petrochenkov]

This is something relatively recent, maybe a few weeks or so.

[Mark-Simulacrum]

@alexcrichton Do you by chance recall the RUST_LOG invocation to make cargo print why it's rebuilding bootstrap?

[Mark-Simulacrum]

Found it: @petrochenkov or @SimonSapin, could you run with RUST_LOG=cargo::ops::cargo_rustc::fingerprint=info and report back with the results? You should see messages from cargo as to why it's recompiling unless we're unsetting RUST_LOG somewhere.

[SimonSapin]

I think this only happens when I run ./x.py test without a specific test suite. bootstrap is built once (if necessary) to run the build system, then built once again after tidy runs:

[…]
* wrapping_iter_arith              lib      stable       1.14.0  
* wrapping_neg                     lib      stable       1.10.0  
* wrapping_ref                     lib      stable       1.14.0  
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for bootstrap v0.0.0 (file:///home/simon/rust/src/bootstrap): new (build_helper v0.1.0 (file:///home/simon/rust/src/build_helper)) != old (build_helper v0.1.0 (file:///home/simon/rust/src/build_helper))
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for build_helper v0.1.0 (file:///home/simon/rust/src/build_helper): RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for filetime v0.1.15: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for cfg-if v0.1.2: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for libc v0.2.39: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for lazy_static v0.2.11: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for getopts v0.2.15: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for serde v1.0.29: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for cmake v0.1.29: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for cc v1.0.6: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for toml v0.4.5: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for num_cpus v1.8.0: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for serde_json v1.0.10: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for itoa v0.3.4: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for dtoa v0.4.2: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for num-traits v0.2.1: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for time v0.1.39: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for serde_derive v1.0.29: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for serde_derive_internals v0.20.0: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for proc-macro2 v0.2.3: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for unicode-xid v0.1.0: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for syn v0.12.14: RUSTFLAGS has changed
 INFO 2018-03-23T10:13:44Z: cargo::ops::cargo_rustc::fingerprint: fingerprint error for quote v0.4.2: RUSTFLAGS has changed
   Compiling cc v1.0.6
   Compiling getopts v0.2.15
   Compiling lazy_static v0.2.11
   Compiling serde v1.0.29
   Compiling libc v0.2.39
   Compiling dtoa v0.4.2
   Compiling itoa v0.3.4
   Compiling unicode-xid v0.1.0
   Compiling num-traits v0.2.1
   Compiling cfg-if v0.1.2
   Compiling proc-macro2 v0.2.3
   Compiling time v0.1.39
   Compiling num_cpus v1.8.0
   Compiling filetime v0.1.15
   Compiling build_helper v0.1.0 (file:///home/simon/rust/src/build_helper)
   Compiling cmake v0.1.29
   Compiling quote v0.4.2
   Compiling syn v0.12.14
   Compiling serde_derive_internals v0.20.0
   Compiling serde_derive v1.0.29
   Compiling toml v0.4.5
   Compiling serde_json v1.0.10
   Compiling bootstrap v0.0.0 (file:///home/simon/rust/src/bootstrap)
    Finished dev [unoptimized] target(s) in 8.39 secs
     Running /home/simon/rust/build/bootstrap/debug/deps/bootstrap-39586fcf3cf86d31

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

Building stage0 test artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[…]

(It looks like this second build is done in order to run a grand total of… zero tests.)

As the log say (thanks for the invocation @Mark-Simulacrum), RUSTFLAGS has changed.

cat /proc/$(pgrep cargo)/environ | tr '\0' '\n' | grep RUSTFLAGS

This shows RUSTFLAGS=-Cdebuginfo=2 during the inital bootstrap build and nothing during the testing build. env["RUSTFLAGS"] = "-Cdebuginfo=2" is set in src/bootstrap/bootstrap.py.

[Mark-Simulacrum]

We should just set those rustflags during the second build probably... If someone wants to make a PR, it should be in src/bootstrap/test.rs, under impl Step for Bootstrap.

[SimonSapin]

#49311