feat: adopt host-owned reliability/perf (engine #15) + bump core

[jmlago]

Bumps the core submodule to unhardcoded-engine main (4409f6f, includes the docs rename #17 and the functional #15 "retire the engine's reliability fold") and adapts the host to own reliability/latency/throughput end-to-end.

Why

Engine #15 made reliability/latency/throughput host-owned: the algebra reads success_rate/latency_ms/tok_s only off the candidate, and the engine no longer folds an EMA, seeds them from metrics, or surfaces them in provider_status. Bumping the submodule without this adaptation broke 3 host tests (empty candidate sets, no live perf, 0 recorded calls).

Changes (host)

• _fold_route_outcome: fold reliability + latency + a call count for every route (peer id for marketplace, provider id for partner/gateway), not just marketplace. Mocked calls fold too, so a mocked call is measured like a live one.
• route_reliability: add a per-route observation counter (count / snapshot_counts).
• /x/market perf: rebuild success_rate/latency_ms/calls from the host folds (engine EMA no longer carries them — it keeps price + credits).
• Tests rewritten to the host-owned model: min_tok_s now filters on candidate-stamped tok_s; the concurrency invariant counts host observations; route-reliability folds via _fold_route_outcome.

Result

Full suite: 277 passed (previously 275 + 2 pre-existing reliability failures — those are now green too).

Follow-up (noted, not in this PR)

Throughput (tok_s) is not yet host-measured/stamped on marketplace offers, so min_tok_s policies currently match only candidates with an explicit tok_s. A route_throughput fold mirroring route_latency is the natural next step.

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

• Refactor

  • Unified route reliability and latency metric collection across all call types for consistent measurement.
  • Redesigned performance metric calculation to derive directly from per-route observations.
  • Added per-route observation count tracking to improve metrics accuracy.

• Chores

  • Updated core subproject dependency.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Free

Run ID: 469077ef-7682-4c21-9b4d-41e4e9047833

📥 Commits

Reviewing files that changed from the base of the PR and between 41111c1 and 887c397.

📒 Files selected for processing (7)

• core
• llm_router_host.py
• route_reliability.py
• shim.py
• tests/test_async_concurrency.py
• tests/test_host.py
• tests/test_route_reliability.py

---

📝 Walkthrough

Walkthrough

route_reliability gains per-route observation counts alongside existing success-rate EMA state. LLMRouterHost._resolve_call_async is refactored so mock, override, and hook resolution paths all assign to a single result and then invoke _fold_route_outcome. _fold_route_outcome broadens its reliability and latency observation key to include routes without a peer_id. market_view() in shim.py drops EMA-based performance lookup in favour of weighted aggregation from live route_reliability/route_latency snapshots.

Changes

Route Measurement Unification

Layer / File(s)	Summary
Per-route observation counts in route_reliability route_reliability.py	Adds _counts: dict[str, int], increments it inside observe(), exposes count(key) and snapshot_counts() accessors, and clears _counts in reset() alongside _rates.
Unified _fold_route_outcome and mock path folding llm_router_host.py	Refactors _resolve_call_async so all resolution branches (mock, override, async/sync hook) assign to result and call _fold_route_outcome once. Updates _fold_route_outcome to observe reliability and latency using a key that falls back to provider_id when peer_id is absent; tool capability observation stays peer-scoped.
market_view _perf aggregation from live snapshots shim.py	Replaces the EMA-based _perf lookup with snapshot aggregation: imports route_reliability/route_latency, sums call counts across matching route keys, and derives success_rate and latency_ms as weighted averages, returning None when no calls are recorded.
Tests for fold unification, counts, and shim tests/test_route_reliability.py, tests/test_host.py, tests/test_async_concurrency.py	Replaces async provider-mock reliability tests with direct _fold_route_outcome unit tests. Updates test_streaming_override_path_folds_route_metrics to assert mock-path latency is now folded. Revises min_tok_s test to use stamped tok_s values. Updates concurrency test to reset route_reliability and derive expected counts from snapshot_counts().

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant _resolve_call_async
  participant _fold_route_outcome
  participant route_reliability
  participant route_latency

  Caller->>_resolve_call_async: request (mock/override/hook)
  rect rgba(100, 149, 237, 0.5)
    Note over _resolve_call_async: All paths now assign to result
    _resolve_call_async->>_resolve_call_async: result = mock/override/hook response
  end
  _resolve_call_async->>_fold_route_outcome: (request, result)
  _fold_route_outcome->>route_reliability: observe(provider|family|peer_or_provider, ok)
  _fold_route_outcome->>route_latency: observe(provider|family|peer_or_provider, latency_ms)
  _fold_route_outcome-->>_resolve_call_async: done
  _resolve_call_async-->>Caller: result

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

A rabbit once measured each hop with great care,
But mocks snuck past counters—that hardly seemed fair!
Now every route folds, from mock all the way,
Snapshots replace EMAs without delay.
🐇 Each hop leaves its mark in the count map today!

---

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Pro by visiting https://app.coderabbit.ai/login.

_{Comment @coderabbitai help to get the list of available commands and usage tips.}