Add cmsis_nn pip package to executorch installation

[Erik-Lundell]

It is needed for the Cortex-M AoT flow.

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18940

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Cancelled Jobs

As of commit 1511f23 with merge base 490ec5c ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / windows / windows-job (gh)
  Process completed with exit code 1.
• pull / unittest-editable / windows / windows-job (gh)
  Process completed with exit code 1.
• trunk / test-models-macos-cpu (resnet50, xnnpack-quantization-delegation) / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• trunk / test-models-macos-mps / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• trunk / unittest-release / linux / linux-job (gh)
  examples/models/test/test_export.py::ExportTest::test_mv3_export_to_executorch
• trunk / unittest-release / windows / windows-job (gh)
  Process completed with exit code 1.

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• trunk / test-models-macos-coreml (mv3) / macos-job (gh)
• trunk / test-selective-build-macos (cmake) / macos-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Erik-Lundell]

macOS builds of cmsis_nn are not doing well... But what do you think about adding cmsis_nn as a dependency like this in general? @digantdesai

[rascani]

Rather than a project wide dependency, I think we should move this to a cortex-m specific requirements file, similar to how we have backends/arm/requirements-arm-ethos-u.txt. We'll probably need to update the cortex-m documentation to reference this extra step.

[Erik-Lundell]

The issue here is that this will be required for AOT, so requiring something like that will either break the Cortex-M backend for pip installation completely, or at least disable the buffer planning functionality. My other proposal is to add it as an optional dependency: pip install executorch[cortex_m]. Or, thirdly, built it into the executorch wheel. WDYT?

[zingo]

@rascani we have the same thing going on for the TOSA arm-backends now when all tools needed to AOT is pip installable (is about a week ago) and we have an old wish from @digantdesai (and also from us/Arm) to make it more part of the executorch pip install process and easier to use. So whatever we decide/do here we will probably do the same for Ethos-U/VGF backends soonish. (before 1.3 ).

[Erik-Lundell]

There is a slight difference in that tosa-tools is available from pypi while cmsis_nn needs building. But my original point still stands imo.

[rascani]

I lean towards the optional dependency approach (pip install executorch[cortex_m]).

Additionally, we should ensure we keep the SHA in sync with the SHA in CMakeLists.txt that is used for fetching the source.

I think it would also be a good idea to publish a pip package for this.

[zingo]

@digantdesai would this be a good solution for Arm (and other) backends also.
e.g.
pip install executorch[cortex_m]
pip install executorch[ethos_u]
pip install executorch[vgf]
pip install executorch[nxp]
pip install executorch[qcom]
...
We will probably think about our names a bit, maybe we end up with a common for ethos_u & vgf like tosa_backend or arm_backends there are a lot of common deps but I also think the devs in most cases not overlapping and 2 packages with clear names might be easier to understand, lets see. Any preferences are welcome as a discussion.

[Erik-Lundell]

I agree, I will push an updated PR proposing this

[Erik-Lundell]

Doing this here #19149 which the cmsis_nn dependency can then build on. Closing this PR.