fix: attribute error/aux rows to the requested model, not 'unknown' - #82
Conversation
Failed and non-chat requests carry no served model_family (it comes from the upstream x_router, absent on error), so every one fell into a single opaque 'unknown' family bucket — 857 requests / 20% err on the prod dashboard with no idea which model they targeted. Fall back to the REQUESTED model in all three aggregation paths, kept in lockstep (the SQL-vs-Python parity test enforces it): - host_store.usage_aggregate family_k (persistent all-time view) - auth_proxy._stats live _stats family buckets (recent feed) - auth_proxy._aggregate_usage_rows (historical replay fold) Served family stays the primary key for 2xx rows; the stored model_family column is untouched (still null on error). New metering test pins the fallback; full suite 501 passed.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughUsage aggregation now attributes error and auxiliary rows without ChangesUsage attribution
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f0d1935977
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # x_router, absent on failure) — fall back to the model the caller ASKED | ||
| # for so failures attribute to a real model instead of piling into a single | ||
| # opaque "unknown" bucket. Served family stays the primary key for 2xx rows. | ||
| " COALESCE(NULLIF(model_family,''),NULLIF(requested_model,''),'unknown') AS family_k," |
There was a problem hiding this comment.
Use the fallback key when filtering model buckets
When a window has error/aux rows with model_family NULL and requested_model='gpt-5.5', this new family_k puts gpt-5.5 into filter_options.models; selecting that model sends model=gpt-5.5 back to /dashboard/api/stats, but host_store._usage_where(..., model_family=model) still filters on raw model_family = %s. Those same fallback-attributed rows are then excluded from totals/recent, so drilling into the new bucket shows too few requests or a lower error rate. Please make the model filter use the same fallback expression, or avoid exposing fallback-only keys as filter options.
Useful? React with 👍 / 👎.
From the 2026-07-15 dashboard review: the "By model family" panel showed 857 requests / 20% err in a single opaque
unknownbucket. Cause: failed and non-chat requests carry no servedmodel_family(it comes from the upstreamx_router, absent on error), so they all collapsed tounknownwith no idea which model they targeted.requested_modelwas captured (auth_proxy.py:3337) but never used as a fallback.Fix
Fall back to the requested model when the served family is absent, in all three aggregation paths — kept in lockstep (the SQL-vs-Python parity test enforces it):
host_store.usage_aggregatefamily_k— persistent all-time viewauth_proxy._statslive family buckets — recent feedauth_proxy._aggregate_usage_rows— historical replay foldServed family stays the primary key for 2xx rows; the stored
model_familycolumn is untouched (still null on error) — only the display bucketing changes.policy_ir/flow_irrouted requests now show under their route label instead ofunknowntoo.Tests
New
test_error_rows_attribute_to_requested_model_not_unknownpins it; full suite 501 passed, 2 skipped.Deploy
Ships with the pending router deploy (main
6f80677, devops).Summary by CodeRabbit