Tracking Issue for Incr. Comp. Regression Testing

[michaelwoerister]

Incremental compilation needs good regression testing, which can be split into the following components:

• End-to-end tests for specific common cases, like already present to some extent in test/incremental. These will use the #[rustc_clean]/#[rustc_dirty] and #![rustc_partition_reused] infrastructure.
• Tests for the Incremental Compilation Hash (ICH) that we compute for input data (i.e. HIR and imported Metadata). I'll open individual issues for each of the below soon.
  • Test ICH for structs (see PR ICH: Add ability to test the ICH of exported metadata items. #36370)
  • Test ICH for enums (incr.comp.: Create Test Case for Incr. Comp. Hash for enums #36674)
  • Test ICH for traits (incr.comp.: Create Test Case for Incr. Comp. Hash for traits #36681)
  • Test ICH for function interfaces (incr.comp.: Create Test Case for Incr. Comp. Hash for function interfaces #36812)
  • Test ICH for function bodies
    • Test ICH for unary and binary expressions (incr.comp.: Create Test Case for Incr. Comp. Hash for unary and binary expressions #37520)
    • Test ICH for call expression (ICH: Add test case for call expressions. #37693)
    • Test ICH for let statements (ICH: Add tests for let- and match-expressions. #37654)
    • Test ICH for match expressions (ICH: Add tests for let- and match-expressions. #37654)
    • Test ICH for if expressions (ICH: Add test case for if- and if-let-expressions. #37694)
    • Test ICH for while loops and loop loops (ICH: Add regression tests for various kinds of loops. #37841)
    • Test ICH for for loops (ICH: Add regression tests for various kinds of loops. #37841)
    • Test ICH for closure expressions (Some incr. comp. hash tests #38202)
    • Test ICH for struct constructor expressions
    • Test ICH for enum constructor expressions (Some incr. comp. hash tests #38202)
    • Test ICH for slice/indexing expressions (Some incr. comp. hash tests #38202)
    • Test ICH for inline assembly (incr. comp.: Improve InlineAsm hashing and add test case #38557)
  • Test ICH for extern blocks (ICH: Fix and test foreign mod hashing. #38479)
  • Test ICH for inherent impls/methods (ICH: Add test cases for inherent impls. #38502)
  • Test ICH for trait impls/methods (ICH: Add some more test cases for trait impls. #38849)
  • Test ICH for type definitions (ICH: Add test case for type alias definitions #37799)
  • Test ICH for statics (incr.comp.: Create Test Case for Incr. Comp. Hash for static declarations #37001)
  • Test ICH for constants (incr.comp.: Create Test Case for Incr. Comp. Hash for const declarations #37000)
  • Test ICH for command line arguments
• Dependency graph oriented tests that make sure that certain edges are present or not present under certain circumstances. These will use the #[rustc_if_this_changed]/#[rustc_then_this_would_need] infrastructure.
• Git history based tests that will walk the history of some selected crates, building each revision incrementally and from scratch and then comparing the two for equality. These tests will be based on cargo incremental. See Continuous Regression Testing for Incremental Compilation #36809 for more information.

In order to make it easy to contribute in this area, we'll also want to

• Compile links to the appropriate documentation of the test framework and some of the concepts and inner workings of incremental compilation
• Provide well-documented examples of what test cases in various areas should look like
• Expand the above list with as concrete as possible descriptions of individual test cases we are interested in, and open individual GH issues for them.