Skip to content

Avoid parsing unchanged lockfiles - #17301

Merged
weihanglo merged 1 commit into
rust-lang:masterfrom
charliermarsh:charlie/cargo-lockfile-equality-fast-path
Aug 3, 2026
Merged

Avoid parsing unchanged lockfiles#17301
weihanglo merged 1 commit into
rust-lang:masterfrom
charliermarsh:charlie/cargo-lockfile-equality-fast-path

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

What does this PR try to resolve?

Previously, checking an unchanged lockfile with --locked or --frozen deserialized both lockfiles and rebuilt their resolution graphs before checking whether their contents already matched. For large workspaces, this repeats expensive work on the common unchanged-lockfile path.

This PR moves the existing line-by-line equality check ahead of the semantic comparison. The existing line-ending behavior is preserved, and differing lockfiles still fall back to the full deserialization and resolution comparison.

On my M4, with 15 interleaved runs of cargo metadata --offline --locked --format-version 1 per workspace:

Workspace Before After Improvement
Cargo 157.209 ms 152.477 ms 3.01%
Ruff 161.157 ms 156.841 ms 2.68%
uv 208.884 ms 201.747 ms 3.42%
Codex 458.177 ms 442.278 ms 3.47%

How to test and review this PR?

The change only reorders the existing equality checks. All 24 lockfile compatibility integration tests pass, including coverage for frozen lockfile preservation and --locked errors.

@rustbot rustbot added the A-lockfile Area: Cargo.lock issues label Aug 2, 2026
Comment thread src/ops/lockfile.rs

@weihanglo weihanglo Aug 3, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. We actually had a similar PR (#17294) recently and got closed because it was a full bot automation not addressing reviewers' question correctly.

This seems to be a fairly pure refactor. Is there any reason it is still behind draft?

View changes since the review

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am also interested in how you fount this, if you don't mind sharing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, sorry, took it out of draft!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tend to use Ultra mode to do exploration of projects and find interesting optimization opportunities, then will go deep on a few of them (try out a prototype, do some benchmarking, etc.). In this case I actually asked it to find opportunities "similar in spirit" to xmakro's Clippy optimizations which I found neat. It's a very different project though with very different bottlenecks so most of the surfaced findings were not that similar in spirit. I'm happy to share the full output if you want to see it (though it is of course LLM-generated).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a fun exploration!

We have a Performance label tracking those. One of the most possible area to improve is asyncifying Cargo. Cargo has its bespoke concurrency control before async/await. For example, dependency source update and manifest parsing are still serial. In #t-cargo > Parallelizing parts of cargo @Kobzol did an experiment on that but not yet ready.

Anyway, if you have found something interesting, feel free to open issues!

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rust project may have an LLM policy soonish I guess, thoughif people bring a tidy-up doc and discuss with me as a human, personally I don't mind it is LLM-assisted or not.

@charliermarsh
charliermarsh marked this pull request as ready for review August 3, 2026 00:34
@rustbot rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 3, 2026
@rustbot

rustbot commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Thanks for the pull request, and welcome! The Rust Project is excited to review your changes, and you should hear from @epage (or someone else) some time within the next two weeks.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

  • @rustbot author: the review is finished, PR author should check the comments and take action accordingly
  • @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue
Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: @epage, @weihanglo
  • @epage, @weihanglo expanded to epage, weihanglo
  • Random selection from epage, weihanglo

@weihanglo weihanglo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@weihanglo
weihanglo added this pull request to the merge queue Aug 3, 2026
Merged via the queue into rust-lang:master with commit 614ec56 Aug 3, 2026
29 checks passed
@rustbot rustbot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 3, 2026
rust-bors Bot pushed a commit to rust-lang/rust that referenced this pull request Aug 5, 2026
Update cargo submodule

21 commits in 7c83d4cc0953b81d823e47d640c64da9b8bd4fac..c79e8f89441b3e73d6d65d125c0c745792808c74
2026-07-29 21:34:53 +0000 to 2026-08-04 19:17:33 +0000
- fix(diag): Ensure diagnostic titles work without snippets (rust-lang/cargo#17304)
- refactor: Remove unnecessary mut in sources (rust-lang/cargo#17305)
- feat(trim-paths): emit unremap files for final artifacts (rust-lang/cargo#17303)
- fix: prevent panic when `package.build` is empty (rust-lang/cargo#17268)
- Add a suggestion when adding `[lints]` to a workspace to use `[workspace.lints]` instead (rust-lang/cargo#17300)
- chore(deps): update embarkstudios/cargo-deny-action action to v2.1.1 (rust-lang/cargo#17291)
- refactor: move sysroot lookup to GlobalContext (rust-lang/cargo#17276)
- fix(trim-paths): unambiguous and reversible remap rules (rust-lang/cargo#17302)
- Avoid parsing unchanged lockfiles (rust-lang/cargo#17301)
- Remove unnecessary to_path_buf (rust-lang/cargo#17295)
- chore(deps): update cargo-semver-checks to v0.50.0 (rust-lang/cargo#17297)
- chore(deps): update actions/checkout action to v6.1.0 (rust-lang/cargo#17290)
- Remove unnecessary return at end of functions (rust-lang/cargo#17292)
- make __CARGO_TEST_FORCE_ARGFILE available in distributed builds (rust-lang/cargo#17293)
- Fix manual_readme lint for lower-priority README files (rust-lang/cargo#17208)
- fix(git): make checkout names independent of git config (rust-lang/cargo#17289)
- fix(diag): Rename redundant_readme to manual_readme (rust-lang/cargo#17288)
- Remove redundant double call .to_string() (rust-lang/cargo#17286)
- fix(completions): complete paths for cargo run arguments (rust-lang/cargo#17284)
- test(git): exercise multi git revision lockfile (rust-lang/cargo#17279)
- add context to lints documentation (rust-lang/cargo#17273)

r? ghost
rust-bors Bot pushed a commit to rust-lang/rust that referenced this pull request Aug 6, 2026
Update cargo submodule



21 commits in 7c83d4cc0953b81d823e47d640c64da9b8bd4fac..c79e8f89441b3e73d6d65d125c0c745792808c74
2026-07-29 21:34:53 +0000 to 2026-08-04 19:17:33 +0000
- fix(diag): Ensure diagnostic titles work without snippets (rust-lang/cargo#17304)
- refactor: Remove unnecessary mut in sources (rust-lang/cargo#17305)
- feat(trim-paths): emit unremap files for final artifacts (rust-lang/cargo#17303)
- fix: prevent panic when `package.build` is empty (rust-lang/cargo#17268)
- Add a suggestion when adding `[lints]` to a workspace to use `[workspace.lints]` instead (rust-lang/cargo#17300)
- chore(deps): update embarkstudios/cargo-deny-action action to v2.1.1 (rust-lang/cargo#17291)
- refactor: move sysroot lookup to GlobalContext (rust-lang/cargo#17276)
- fix(trim-paths): unambiguous and reversible remap rules (rust-lang/cargo#17302)
- Avoid parsing unchanged lockfiles (rust-lang/cargo#17301)
- Remove unnecessary to_path_buf (rust-lang/cargo#17295)
- chore(deps): update cargo-semver-checks to v0.50.0 (rust-lang/cargo#17297)
- chore(deps): update actions/checkout action to v6.1.0 (rust-lang/cargo#17290)
- Remove unnecessary return at end of functions (rust-lang/cargo#17292)
- make __CARGO_TEST_FORCE_ARGFILE available in distributed builds (rust-lang/cargo#17293)
- Fix manual_readme lint for lower-priority README files (rust-lang/cargo#17208)
- fix(git): make checkout names independent of git config (rust-lang/cargo#17289)
- fix(diag): Rename redundant_readme to manual_readme (rust-lang/cargo#17288)
- Remove redundant double call .to_string() (rust-lang/cargo#17286)
- fix(completions): complete paths for cargo run arguments (rust-lang/cargo#17284)
- test(git): exercise multi git revision lockfile (rust-lang/cargo#17279)
- add context to lints documentation (rust-lang/cargo#17273)

r? ghost
@rustbot rustbot added this to the 1.99.0 milestone Aug 6, 2026
pull Bot pushed a commit to LeeeeeeM/miri that referenced this pull request Aug 8, 2026
Update cargo submodule



21 commits in 7c83d4cc0953b81d823e47d640c64da9b8bd4fac..c79e8f89441b3e73d6d65d125c0c745792808c74
2026-07-29 21:34:53 +0000 to 2026-08-04 19:17:33 +0000
- fix(diag): Ensure diagnostic titles work without snippets (rust-lang/cargo#17304)
- refactor: Remove unnecessary mut in sources (rust-lang/cargo#17305)
- feat(trim-paths): emit unremap files for final artifacts (rust-lang/cargo#17303)
- fix: prevent panic when `package.build` is empty (rust-lang/cargo#17268)
- Add a suggestion when adding `[lints]` to a workspace to use `[workspace.lints]` instead (rust-lang/cargo#17300)
- chore(deps): update embarkstudios/cargo-deny-action action to v2.1.1 (rust-lang/cargo#17291)
- refactor: move sysroot lookup to GlobalContext (rust-lang/cargo#17276)
- fix(trim-paths): unambiguous and reversible remap rules (rust-lang/cargo#17302)
- Avoid parsing unchanged lockfiles (rust-lang/cargo#17301)
- Remove unnecessary to_path_buf (rust-lang/cargo#17295)
- chore(deps): update cargo-semver-checks to v0.50.0 (rust-lang/cargo#17297)
- chore(deps): update actions/checkout action to v6.1.0 (rust-lang/cargo#17290)
- Remove unnecessary return at end of functions (rust-lang/cargo#17292)
- make __CARGO_TEST_FORCE_ARGFILE available in distributed builds (rust-lang/cargo#17293)
- Fix manual_readme lint for lower-priority README files (rust-lang/cargo#17208)
- fix(git): make checkout names independent of git config (rust-lang/cargo#17289)
- fix(diag): Rename redundant_readme to manual_readme (rust-lang/cargo#17288)
- Remove redundant double call .to_string() (rust-lang/cargo#17286)
- fix(completions): complete paths for cargo run arguments (rust-lang/cargo#17284)
- test(git): exercise multi git revision lockfile (rust-lang/cargo#17279)
- add context to lints documentation (rust-lang/cargo#17273)

r? ghost
github-actions Bot pushed a commit to rust-lang/stdarch that referenced this pull request Aug 10, 2026
Update cargo submodule



21 commits in 7c83d4cc0953b81d823e47d640c64da9b8bd4fac..c79e8f89441b3e73d6d65d125c0c745792808c74
2026-07-29 21:34:53 +0000 to 2026-08-04 19:17:33 +0000
- fix(diag): Ensure diagnostic titles work without snippets (rust-lang/cargo#17304)
- refactor: Remove unnecessary mut in sources (rust-lang/cargo#17305)
- feat(trim-paths): emit unremap files for final artifacts (rust-lang/cargo#17303)
- fix: prevent panic when `package.build` is empty (rust-lang/cargo#17268)
- Add a suggestion when adding `[lints]` to a workspace to use `[workspace.lints]` instead (rust-lang/cargo#17300)
- chore(deps): update embarkstudios/cargo-deny-action action to v2.1.1 (rust-lang/cargo#17291)
- refactor: move sysroot lookup to GlobalContext (rust-lang/cargo#17276)
- fix(trim-paths): unambiguous and reversible remap rules (rust-lang/cargo#17302)
- Avoid parsing unchanged lockfiles (rust-lang/cargo#17301)
- Remove unnecessary to_path_buf (rust-lang/cargo#17295)
- chore(deps): update cargo-semver-checks to v0.50.0 (rust-lang/cargo#17297)
- chore(deps): update actions/checkout action to v6.1.0 (rust-lang/cargo#17290)
- Remove unnecessary return at end of functions (rust-lang/cargo#17292)
- make __CARGO_TEST_FORCE_ARGFILE available in distributed builds (rust-lang/cargo#17293)
- Fix manual_readme lint for lower-priority README files (rust-lang/cargo#17208)
- fix(git): make checkout names independent of git config (rust-lang/cargo#17289)
- fix(diag): Rename redundant_readme to manual_readme (rust-lang/cargo#17288)
- Remove redundant double call .to_string() (rust-lang/cargo#17286)
- fix(completions): complete paths for cargo run arguments (rust-lang/cargo#17284)
- test(git): exercise multi git revision lockfile (rust-lang/cargo#17279)
- add context to lints documentation (rust-lang/cargo#17273)

r? ghost
github-actions Bot pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Aug 10, 2026
Update cargo submodule



21 commits in 7c83d4cc0953b81d823e47d640c64da9b8bd4fac..c79e8f89441b3e73d6d65d125c0c745792808c74
2026-07-29 21:34:53 +0000 to 2026-08-04 19:17:33 +0000
- fix(diag): Ensure diagnostic titles work without snippets (rust-lang/cargo#17304)
- refactor: Remove unnecessary mut in sources (rust-lang/cargo#17305)
- feat(trim-paths): emit unremap files for final artifacts (rust-lang/cargo#17303)
- fix: prevent panic when `package.build` is empty (rust-lang/cargo#17268)
- Add a suggestion when adding `[lints]` to a workspace to use `[workspace.lints]` instead (rust-lang/cargo#17300)
- chore(deps): update embarkstudios/cargo-deny-action action to v2.1.1 (rust-lang/cargo#17291)
- refactor: move sysroot lookup to GlobalContext (rust-lang/cargo#17276)
- fix(trim-paths): unambiguous and reversible remap rules (rust-lang/cargo#17302)
- Avoid parsing unchanged lockfiles (rust-lang/cargo#17301)
- Remove unnecessary to_path_buf (rust-lang/cargo#17295)
- chore(deps): update cargo-semver-checks to v0.50.0 (rust-lang/cargo#17297)
- chore(deps): update actions/checkout action to v6.1.0 (rust-lang/cargo#17290)
- Remove unnecessary return at end of functions (rust-lang/cargo#17292)
- make __CARGO_TEST_FORCE_ARGFILE available in distributed builds (rust-lang/cargo#17293)
- Fix manual_readme lint for lower-priority README files (rust-lang/cargo#17208)
- fix(git): make checkout names independent of git config (rust-lang/cargo#17289)
- fix(diag): Rename redundant_readme to manual_readme (rust-lang/cargo#17288)
- Remove redundant double call .to_string() (rust-lang/cargo#17286)
- fix(completions): complete paths for cargo run arguments (rust-lang/cargo#17284)
- test(git): exercise multi git revision lockfile (rust-lang/cargo#17279)
- add context to lints documentation (rust-lang/cargo#17273)

r? ghost
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-lockfile Area: Cargo.lock issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants