Massive compiler perf regression

[Gankra]

First reported on Reddit here. See that post for more details and examples.

This crate has gone from a build time of 17 seconds with the compiler consuming 168MB of RAM at peak to 320 secs and 1670MB. Apparently this regression occurred somewhere between jan 15-18, and is in the current beta. The author hypothesizes that it may have to do with complex usage of traits.

[dirk]

Quoting a comment from Reddit (see here) so it doesn't get lost:

Paging /u/arielby and /u/nikomatsakis, they always know what's going on in the type checker

EDIT: Does the Jan 18 nightly contain the obligation forest (#30533), by any chance?

[aturon]

cc @nikomatsakis @arielb1

[White-Oak]

+1 on this (for reference output of cargo rustc -- -Z time-passes). When talking about usage of complex traits (example, once again, here), it was always been slow, but on an acceptible rate. The author of library even goes about that in a README.md:

Be aware, due to an ongoing issue with rustc, the compile time of your code will exponentially increase with the complexity of your parsers. In practice I've found things get bad after about 10 combinations or so. You can get around this by boxing a parser...

The issue he mentioned is #22204. This (current) may be considered duplicate of that, though, again, the problem was not of the current magnitude.

The author considers boxing to help:

This "flattens" the type signature of the parser into a trait object, which will improve compile-time at the cost of runtime performance due to dynamic dispath.

But, once again, it doesn't help much now.

[White-Oak]

Ah, as well, this may be considered as a regression from stable to beta:
Code compiles rather quickly on stable 1.6, but slowly on beta 1.7. I just went further to find exact nightlies, which differ in compilation times.

[gereeter]

Note that the obligation forest PR was merged on January 16, so it did get merged in the window in which the regression happened.

[nikomatsakis]

Seems pretty likely to have to do with the obligation forest and also with the change in caching behavior that accompanied it. I can investigate some.

[MagaTailor]

Is the syntax_syntex issue related? It was always bad but now it's definitely worse.

[nikomatsakis]

I can certainly confirm that this is traits-related; at least, the crate in question spends about 85% of its compilation time in middle::traits. I'm looking more in depth now.

[nikomatsakis]

@petevine unlikely to be related to that, but I don't know

[nikomatsakis]

Seems pretty clear that the caching of sized results is no longer kicking in like it once did. This probably affects the script crate from servo too.

[nikomatsakis]

e.g., we repeatedly prove that T: Sized for the following types T. Here, the first number is the number of times we have proved it in my current logs. Note that compilation did not yet complete. :)

  79297 <*const alloc::rc::RcBox<fn() -> Box<peruse::parsers::Parser<I=[grammar_lexer::Token], O=grammar::Expr>> {parser::program::expression}>
  81467 <core::marker::PhantomData<alloc::rc::RcBox<fn() -> Box<peruse::parsers::Parser<I=[grammar_lexer::Token], O=grammar::Expr>> {parser::program::expression}>>
  81467 <core::nonzero::NonZero<*const alloc::rc::RcBox<fn() -> Box<peruse::parsers::Parser<I=[grammar_lexer::Token], O=grammar::Expr>> {parser::program::expression}>>
  82807 <core::ptr::Shared<alloc::rc::RcBox<fn() -> Box<peruse::parsers::Parser<I=[grammar_lexer::Token], O=grammar::Expr>> {parser::program::expression}>>
  82984 <peruse::parsers::RecursiveParser<[grammar_lexer::Token], grammar::Expr, fn() -> Box<peruse::parsers::Parser<I=[grammar_lexer::Token], O=grammar::Expr>> {parser::program::expression}>
  82999 <alloc::rc::Rc<fn() -> Box<peruse::parsers::Parser<I=[grammar_lexer::Token], O=grammar::Expr>> {parser::program::expression}>
 132273 <fn() -> Box<peruse::parsers::Parser<I=[grammar_lexer::Token], O=grammar::Expr>> {parser::program::expression}