Skip to content

Integrate ONNX Runtime for local model execution #61

@KSemenenko

Description

@KSemenenko

Problem

The product needs a second local runtime path for models that fit an ONNX-based execution strategy.

Scope

  • Track CPU and GPU-capable ONNX Runtime integration at the product-contract level
  • Cover execution-provider visibility, compatibility, and operator-facing state
  • Keep the runtime aligned with the shared local session model

Out of scope

  • LLamaSharp implementation
  • Provider toolchain readiness for external CLIs

Implementation notes

  • Use ONNX Runtime as the approved local ONNX execution layer
  • Keep execution-provider reporting explicit
  • Align the issue with the model-vault and mixed-session flows

Definition of Done

  • The issue defines the ONNX runtime expectations clearly
  • Later implementation can proceed without ambiguity about product behavior

Verification

  • Review the issue against the local-runtime abstraction and model-vault issue

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions