Skip to content

Save/restore more items in cache with incremental compilation#123221

Merged
bors merged 2 commits into
rust-lang:masterfrom
pacak:cache_emit
Apr 7, 2024
Merged

Save/restore more items in cache with incremental compilation#123221
bors merged 2 commits into
rust-lang:masterfrom
pacak:cache_emit

Conversation

@pacak

@pacak pacak commented Mar 30, 2024

Copy link
Copy Markdown
Contributor

Right now they don't play very well together, consider a simple example:

$ export RUSTFLAGS="--emit asm"
$ cargo new --lib foo
     Created library `foo` package
$ cargo build -q
$ touch src/lib.rs
$ cargo build
error: could not copy 
  "/path/to/foo/target/debug/deps/foo-e307cc7fa7b6d64f.4qbzn9k8mosu50a5.rcgu.s" 
  to "/path/to/foo/target/debug/deps/foo-e307cc7fa7b6d64f.s": 
  No such file or directory (os error 2)

Touch triggers the rebuild, incremental compilation detects no changes (yay) and everything explodes while trying to copy files were they should go.

This pull request fixes it by copying and restoring more files in the incremental compilation cache

Fixes #89149
Fixes #88829

Related: https://internals.rust-lang.org/t/interaction-between-incremental-compilation-and-emit/20551

@rustbot

rustbot commented Mar 30, 2024

Copy link
Copy Markdown
Collaborator

r? @Nadrieril

rustbot has assigned @Nadrieril.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Mar 30, 2024
@rustbot

rustbot commented Mar 30, 2024

Copy link
Copy Markdown
Collaborator

Some changes occurred in run-make tests.

cc @jieyouxu

Some changes occurred in compiler/rustc_codegen_gcc

cc @antoyo, @GuillaumeGomez

Some changes occurred in compiler/rustc_codegen_cranelift

cc @bjorn3

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@Nadrieril

Copy link
Copy Markdown
Member

r? compiler

@rustbot rustbot assigned fmease and unassigned Nadrieril Mar 30, 2024
Comment thread compiler/rustc_codegen_ssa/src/lib.rs Outdated
Comment thread compiler/rustc_codegen_ssa/src/lib.rs Outdated
Comment thread tests/run-make/asm-incr-cache/rmake.rs Outdated
Comment thread tests/run-make/asm-incr-cache/rmake.rs Outdated
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

Comment thread tests/run-make/asm-incr-cache/rmake.rs Outdated
Comment thread tests/run-make/asm-incr-cache/rmake.rs Outdated

@fmease fmease left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Overall looks good. I have a couple of minor nitpicks

Comment thread compiler/rustc_codegen_ssa/src/back/write.rs Outdated
Comment thread compiler/rustc_codegen_ssa/src/back/write.rs Outdated
Comment thread compiler/rustc_codegen_ssa/src/lib.rs Outdated
Comment thread tests/run-make/asm-incr-cache/lib.rs
Comment thread tests/run-make/asm-incr-cache/rmake.rs Outdated

@jieyouxu jieyouxu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The run-make test and support library changes look good to me, just left some minor nits 👍

@fmease fmease added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 6, 2024
@rust-log-analyzer

This comment has been minimized.

@pacak

pacak commented Apr 6, 2024

Copy link
Copy Markdown
Contributor Author

🎉

@matthiaskrgr

Copy link
Copy Markdown
Member

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Apr 6, 2024
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 6, 2024
Save/restore more items in cache with incremental compilation

Right now they don't play very well together, consider a simple example:

```
$ export RUSTFLAGS="--emit asm"
$ cargo new --lib foo
     Created library `foo` package
$ cargo build -q
$ touch src/lib.rs
$ cargo build
error: could not copy
  "/path/to/foo/target/debug/deps/foo-e307cc7fa7b6d64f.4qbzn9k8mosu50a5.rcgu.s"
  to "/path/to/foo/target/debug/deps/foo-e307cc7fa7b6d64f.s":
  No such file or directory (os error 2)
```

Touch triggers the rebuild, incremental compilation detects no changes (yay) and everything explodes while trying to copy files were they should go.

This pull request fixes it by copying and restoring more files in the incremental compilation cache

Fixes rust-lang#89149
Fixes rust-lang#88829

Related: https://internals.rust-lang.org/t/interaction-between-incremental-compilation-and-emit/20551
@bors

bors commented Apr 6, 2024

Copy link
Copy Markdown
Collaborator

⌛ Trying commit 691e953 with merge a0e26ff...

@bors

bors commented Apr 6, 2024

Copy link
Copy Markdown
Collaborator

☀️ Try build successful - checks-actions
Build commit: a0e26ff (a0e26ff7cff359f39a275ec3ef2b56a070a4d94d)

1 similar comment
@bors

bors commented Apr 6, 2024

Copy link
Copy Markdown
Collaborator

☀️ Try build successful - checks-actions
Build commit: a0e26ff (a0e26ff7cff359f39a275ec3ef2b56a070a4d94d)

@rust-timer

This comment has been minimized.

@rust-timer

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (a0e26ff): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 666.761s -> 669.173s (0.36%)
Artifact size: 318.27 MiB -> 318.28 MiB (0.00%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Apr 6, 2024
@pacak

pacak commented Apr 6, 2024

Copy link
Copy Markdown
Contributor Author

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 6, 2024
@fmease

fmease commented Apr 7, 2024

Copy link
Copy Markdown
Member

Thanks!

@bors r=fmease,jieyouxu

@bors

bors commented Apr 7, 2024

Copy link
Copy Markdown
Collaborator

📌 Commit 691e953 has been approved by fmease,jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 7, 2024
@bors

bors commented Apr 7, 2024

Copy link
Copy Markdown
Collaborator

⌛ Testing commit 691e953 with merge fc1a4c5...

@bors

bors commented Apr 7, 2024

Copy link
Copy Markdown
Collaborator

☀️ Test successful - checks-actions
Approved by: fmease,jieyouxu
Pushing fc1a4c5 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Apr 7, 2024
@bors bors merged commit fc1a4c5 into rust-lang:master Apr 7, 2024
@rustbot rustbot added this to the 1.79.0 milestone Apr 7, 2024
@rust-timer

Copy link
Copy Markdown
Collaborator

Finished benchmarking commit (fc1a4c5): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-3.5% [-3.6%, -3.5%] 2
Improvements ✅
(secondary)
-2.5% [-4.3%, -0.7%] 25
All ❌✅ (primary) -3.5% [-3.6%, -3.5%] 2

Cycles

This benchmark run did not return any relevant results for this metric.

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 667.68s -> 667.899s (0.03%)
Artifact size: 318.27 MiB -> 318.21 MiB (-0.02%)

@pacak pacak deleted the cache_emit branch April 7, 2024 14:27
bjorn3 pushed a commit to bjorn3/rust that referenced this pull request Apr 11, 2024
Save/restore more items in cache with incremental compilation

Right now they don't play very well together, consider a simple example:

```
$ export RUSTFLAGS="--emit asm"
$ cargo new --lib foo
     Created library `foo` package
$ cargo build -q
$ touch src/lib.rs
$ cargo build
error: could not copy
  "/path/to/foo/target/debug/deps/foo-e307cc7fa7b6d64f.4qbzn9k8mosu50a5.rcgu.s"
  to "/path/to/foo/target/debug/deps/foo-e307cc7fa7b6d64f.s":
  No such file or directory (os error 2)
```

Touch triggers the rebuild, incremental compilation detects no changes (yay) and everything explodes while trying to copy files were they should go.

This pull request fixes it by copying and restoring more files in the incremental compilation cache

Fixes rust-lang#89149
Fixes rust-lang#88829

Related: https://internals.rust-lang.org/t/interaction-between-incremental-compilation-and-emit/20551
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failure on subsequent builds when --emit asm used without -C save-temps. --emit=llvm-bc in workspaces

10 participants