Add XNNPACK, MobileNetV2, MobileBERT, Llama, ResNet18 to RISC-V testing matrix

[luhenry]

Summary

This is the continuation of #19399 and #19521, to deliver on Phase 2 of #18991

Test plan

This code is exclusively test code. Everything works out of the box, and CI will validate.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19617

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 2 Active SEVs

There are 2 currently active SEVs. If your PR is affected, please view them below:

• Run pull request jobs on OSDC runners in shadow mode
• Rollout OSDC prod cluster in us-west-1

✅ You can merge normally! (38 Unrelated Failures)

As of commit 215c908 with merge base 7355d7b ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-qnn-delegate-linux / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-qnn-models-linux (dl3) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-qnn-models-linux (mv2) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-qnn-models-linux (mv3) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-qnn-python-imports-linux / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-qnn-testsuite-linux / test-backend-linux (qnn, operators) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• Test QNN Backend / test-qnn / test-backend-linux (qnn, models) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• Test QNN Backend / test-qnn / test-backend-linux (qnn, operators) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, conv_former) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, dl3) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, ic3) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, ic4) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, mb) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, mv2) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, mv3) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, vit) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-model (fp32, w2l) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, albert) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, bert) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, cvt) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, distilbert) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, dit) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, efficientnet) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, focalnet) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, mobilevit_v1) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, mobilevit_v2) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, pvt) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, roberta) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• trunk / test-qnn-optimum-model (fp32, swin) / linux-job (gh) (similar failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-qnn-passes-linux / linux-job (gh) (trunk failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job (gh) (trunk failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-static-llama-qnn-linux (stories_110m) / linux-job (gh) (trunk failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]
• pull / test-static-llama-qnn-linux (stories_260k_bc) / linux-job (gh) (trunk failure)
  /pytorch/executorch/examples/qualcomm/oss_scripts/llama/qnn_multimodal_runner.cpp:185:7: error: type 'float' cannot be narrowed to 'bool' in initializer list [-Wc++11-narrowing]

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[luhenry]

cc @rascani @mergennachin, and @GregoryComer @digantdesai for XNNPACK

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[luhenry]

Do I understand correctly that all the QNN tests are unrelated?