Skip to content

Separate CI jobs, add job caching#357

Merged
cart merged 1 commit into
bevyengine:masterfrom
AngelOnFira:add-separate-ci-jobs
Aug 26, 2020
Merged

Separate CI jobs, add job caching#357
cart merged 1 commit into
bevyengine:masterfrom
AngelOnFira:add-separate-ci-jobs

Conversation

@AngelOnFira

Copy link
Copy Markdown
Contributor

This PR separates CI jobs for better parallelism across runners. There are now 5 separate jobs that run. A build and test for stable and nightly, as well as a "clean" job. Clean includes both cargo fmt and cargo clippy as they are both quciker than normal cargo check or cargo test, and so they will complete around the same time.

As we can see below, there were definitely some issues with adding caching to CI vanilla. There were a few improvements I added:

  • Turn off incremental builds 1, 2. This decreased the size of the cache from about 2GB to 600MB.
  • Added the compiler flag -C debuginfo=0, which decreased the size further. I don't think this will have negative effects with cargo test or cargo check.
  • Only save the target folder. Normally, both the registry and the git folder are saved. These can be redownloaded fast enough, and would just clutter the zip with more small files to open upon downloading the cache.
total time cache size open cache time cargo test time create cache time
stable, no cache 8:20 - - 8:06 -
nightly, no cache 7:50 - - 7:29 -
stable, cache, incremental 9:04 2202 MB 3:34 5:07 ~ 4:00
nightly, cache, incremental 8:56 2186 MB 3:34 4:51 ~ 4:00
stable, cache, non incremental 4:22 586 MB 0:42 3:24 0:23
nightly, cache, non incremental 4:52 600 MB 0:40 3:32 0:22

Likely CI will change further with the addition of more steps, however, this should be a good starting point for separation of jobs. Some unknowns that I will have to examine once this is running on master:

  • How does the cache work once a new nightly is released the day after? The cache won't be busted unless Cargo.lock changes, so another busting method might be needed for nightlies.

@qarmin qarmin mentioned this pull request Aug 26, 2020
8 tasks
@cart

cart commented Aug 26, 2020

Copy link
Copy Markdown
Member

Brilliant work. Thanks!

@cart cart merged commit 4562a09 into bevyengine:master Aug 26, 2020
@AngelOnFira AngelOnFira deleted the add-separate-ci-jobs branch August 28, 2020 10:36
@CleanCut CleanCut mentioned this pull request Sep 2, 2020
mrk-its pushed a commit to mrk-its/bevy that referenced this pull request Oct 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants