Skip to content

rustc: Prepare to enable ThinLTO by default#46382

Merged
bors merged 3 commits into
rust-lang:masterfrom
alexcrichton:thinlto-default
Dec 3, 2017
Merged

rustc: Prepare to enable ThinLTO by default#46382
bors merged 3 commits into
rust-lang:masterfrom
alexcrichton:thinlto-default

Conversation

@alexcrichton

@alexcrichton alexcrichton commented Nov 29, 2017

Copy link
Copy Markdown
Member

This commit almost enables ThinLTO and multiple codegen units in release mode by
default but is blocked on #46346 now before pulling the trigger.

@rust-highfive

Copy link
Copy Markdown
Contributor

r? @arielb1

(rust_highfive has picked a reviewer for you, use r? to override)

@alexcrichton

Copy link
Copy Markdown
Member Author

r? @michaelwoerister

cc @rust-lang/compiler

@sfackler

Copy link
Copy Markdown
Member

Is this a blocker? #46346

@alexcrichton

Copy link
Copy Markdown
Member Author

Are we sure that's related to ThinLTO? I've tested everything locally and wasn't able to reproduce myself.

@alexcrichton

Copy link
Copy Markdown
Member Author

Urgh it may very well be. I swear to god.

@kennytm kennytm added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 29, 2017
Comment thread src/rustllvm/PassWrapper.cpp Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not use a std::unordered_set here.

@cuviper

cuviper commented Nov 30, 2017

Copy link
Copy Markdown
Member

I see that LLVMRustThinLTOAvailable gates on just LLVM_VERSION_GE(4, 0). How well are you expecting this to work when using an external LLVM-4.0?
(IOW, are there rust-lang/llvm patches that I definitely need here?)

@alexcrichton

Copy link
Copy Markdown
Member Author

@cuviper IIRC rust-lang/llvm@e45c75d is the main (and possibly only) ThinLTO related patch (although it fixes a preexisting bug).

In general though I lose track over time what patch was for what on our fork, although everything that matters is upstream in some form so it's just a matter of time until we update to get it.

@cuviper

cuviper commented Nov 30, 2017

Copy link
Copy Markdown
Member

Ok, thanks, we already grabbed that one. I'll look back over the branch again later myself. I do appreciate that everything is fixed upstream first!

@michaelwoerister

Copy link
Copy Markdown
Member

Awesome! Will review shortly. Am I reading this right, this is still blocked on #46346?

@alexcrichton

Copy link
Copy Markdown
Member Author

@michaelwoerister unfortunately yes :(

@michaelwoerister michaelwoerister left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR looks good to me. Let's hope it doesn't bit rot too much until #46346 is resolved.

Can you explain to me why you don't use addPreservedGUID anymore?

Comment thread src/librustc/session/mod.rs Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

compilation compilation

Comment thread src/libstd/sys_common/backtrace.rs Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on -> one

Comment thread src/rustllvm/PassWrapper.cpp Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this too conservative before?

Previously we were too eagerly exporting almost all symbols used in ThinLTO
which can cause a whole host of problems downstream! This commit instead fixes
this error by aligning more closely with `lib/LTO/LTO.cpp` in LLVM's codebase
which is to only change the linkage of summaries which are computed as dead.

Closes rust-lang#46374
Helps to avoid hitting path limits on Windows
This commit prepares to enable ThinLTO and multiple codegen units in release
mode by default. We've still got a debuginfo bug or two to sort out before
actually turning it on by default.
@alexcrichton alexcrichton changed the title rustc: Enable ThinLTO by default rustc: Prepare to enable ThinLTO by default Nov 30, 2017
@alexcrichton

Copy link
Copy Markdown
Member Author

@michaelwoerister ok I've updated the listing of commits and addressed comments. The commits now to not enable ThinLTO by default but instead retain retain today's default. The repo is left in a state though to make it a one line change to turn ThinLTO on by default in the future.

To answer your question about GUIDPreservedSymbols though there's more info on #46374 but it boils down to two things:

  • In the first location where GUIDPreservedSymbols was used the logic of addPreservedGUID wasn't necessary. The same result is computed whether we call addPreservedGUID or not.
  • In the second location it was incorrect to call addPreservedGUID in the sense that it added the transitive closure of symbols to that set. That in turn caused ThinLTO exposes too many symbols #46374 where too many symbols were exported (aka all of them) rather than just the set we wanted. By avoiding addPreservedGUID and tweaking the definition of isExported this bug was fixed though.

@michaelwoerister

Copy link
Copy Markdown
Member

Thanks for the explanation!

@bors r+

@bors

bors commented Nov 30, 2017

Copy link
Copy Markdown
Collaborator

📌 Commit 855f6d1 has been approved by michaelwoerister

@kennytm kennytm added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 30, 2017
@bors

bors commented Dec 2, 2017

Copy link
Copy Markdown
Collaborator

⌛ Testing commit 855f6d1 with merge 088f328...

bors added a commit that referenced this pull request Dec 2, 2017
rustc: Prepare to enable ThinLTO by default

This commit *almost* enables ThinLTO and multiple codegen units in release mode by
default but is blocked on #46346 now before pulling the trigger.
@bors

bors commented Dec 3, 2017

Copy link
Copy Markdown
Collaborator

☀️ Test successful - status-appveyor, status-travis
Approved by: michaelwoerister
Pushing 088f328 to master...

@bors bors merged commit 855f6d1 into rust-lang:master Dec 3, 2017
alexcrichton added a commit to alexcrichton/rust that referenced this pull request Dec 7, 2017
In rust-lang#46382 the logic around linkage preservation with ThinLTO ws tweaked but the
loop that registered all otherwise exported GUID values as "don't internalize
me please" was erroneously too conservative and only asking "external" linkage
items to not be internalized. Instead we actually want the inversion of that
condition, everything *without* "local" linkage to be internalized.

This commit updates the condition there, adds a test, and...

Closes rust-lang#46543
bors added a commit that referenced this pull request Dec 8, 2017
rustc: Further tweak linkage in ThinLTO

In #46382 the logic around linkage preservation with ThinLTO ws tweaked but the
loop that registered all otherwise exported GUID values as "don't internalize
me please" was erroneously too conservative and only asking "external" linkage
items to not be internalized. Instead we actually want the inversion of that
condition, everything *without* "local" linkage to be internalized.

This commit updates the condition there, adds a test, and...

Closes #46543
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants