Add cost accounting extension for Usage events

[elCaptnCode]

Why are these changes needed?

AG2 currently normalizes provider token usage into Usage / UsageEvent, but there is no optional first-class way to derive estimated cost from that usage. Model prices change frequently and provider billing structures vary, so this PR keeps AG2 core usage accounting factual and adds cost accounting as an extension.

This adds ag2.extensions.cost_accounting, which estimates USD cost from UsageEvent data and a maintained pricing catalog. The catalog loader supports LiteLLM-style fields such as:

• input_cost_per_token
• output_cost_per_token
• cache_read_input_token_cost
• cache_creation_input_token_cost
• output_cost_per_reasoning_token

The extension watches UsageEvent, AG2's source of truth for usage reporting, so it avoids double counting model responses and sub-agent rollups.

Included APIs:

• CostCatalog / ModelPricing
• UsageCostEstimator
• CostEstimate
• CostAccountingObserver

Unknown models return pricing_source="not_configured" with a warning instead of guessing.

Related issue number

No existing issue. This PR implements an optional cost accounting extension over AG2 usage telemetry.

Checks

• I've included any doc changes needed for https://docs.ag2.ai/. See https://docs.ag2.ai/latest/docs/contributor-guide/documentation/ to build and test documentation locally.
  • Added ag2/extensions/cost_accounting/README.md because this repository clone does not include a published docs page for this new extension.
• I've added tests (if relevant) corresponding to the changes introduced in this PR.
• I've made sure all auto checks have passed.
  • Current reported GitHub status for the latest commit shows the CLA check passing. I do not have the repo's expected uv / latest ruff toolchain in this local environment, so I also ran focused pytest, compile checks, and a live OpenAI smoke locally.

Validation run locally:

python -m pytest -q test\extensions\cost_accounting\test_cost_accounting.py test\test_usage.py test\observer\test_builtin.py
34 passed

python -m compileall -q ag2\extensions\cost_accounting test\extensions\cost_accounting
passed

Live provider smoke with OpenAIConfig and gpt-4.1-nano:

reply_body 'ok'
prompt_tokens 16
completion_tokens 1
total_tokens 17
cache_read_input_tokens 0
estimated_cost_usd 0.000002000
alerts 1 warning
pricing_source catalog

That smoke used a real AG2 Agent.ask() call with OpenAIConfig, MemoryStream, and CostAccountingObserver attached.

AI assistance

• I understand the changes in this PR and can explain them in my own words.
• I have verified that the PR description accurately reflects the actual diff.
• If AI assistance was used, I reviewed, tested, and validated the generated code/text before submitting.

[elCaptnCode]

Update: I added an integration-style test that runs the extension through Agent.ask() with TestConfig, and I also ran a real provider-backed smoke with OpenAIConfig + gpt-4.1-nano locally.

Focused validation:

python -m pytest -q test\extensions\cost_accounting\test_cost_accounting.py test\test_usage.py test\observer\test_builtin.py
34 passed

python -m compileall -q ag2\extensions\cost_accounting test\extensions\cost_accounting
passed

Live provider smoke:

reply_body 'ok'
prompt_tokens 16
completion_tokens 1
total_tokens 17
cache_read_input_tokens 0
estimated_cost_usd 0.000002000
alerts 1 warning
pricing_source catalog

The live smoke also caught and fixed dated OpenAI model revision lookup (gpt-4.1-nano-2025-04-14 falling back to catalog key openai/gpt-4.1-nano).

[codecov]

Codecov Report

❌ Patch coverage is 83.90244% with 33 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
ag2/extensions/cost_accounting/catalog.py	80.23%	15 Missing and 2 partials ⚠️
ag2/extensions/cost_accounting/estimator.py	86.66%	5 Missing and 5 partials ⚠️
ag2/extensions/cost_accounting/observer.py	85.00%	3 Missing and 3 partials ⚠️

Files with missing lines	Coverage Δ
ag2/extensions/cost_accounting/__init__.py	100.00% <100.00%> (ø)
ag2/extensions/cost_accounting/observer.py	85.00% <85.00%> (ø)
ag2/extensions/cost_accounting/estimator.py	86.66% <86.66%> (ø)
ag2/extensions/cost_accounting/catalog.py	80.23% <80.23%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[vvlrff]

Hi @elCaptnCode, thanks for the PR, but this implementation drops the subagent cost to $0.

[genisis0x]

The mechanics here look solid: Decimal end to end with Decimal(str(value)) to avoid float artifacts, clamping tokens to >= 0, and separating regular input / cache-read / cache-creation / output / reasoning into their own line items is the right granularity for something people will reconcile against a provider invoice.

The one thing I'd want to nail down is the assumption baked into this block, because it's provider-dependent:

cached_input_tokens = min(prompt_tokens, cache_read_tokens + cache_creation_tokens)
regular_input_tokens = prompt_tokens - cached_input_tokens

This only produces the right regular_input_tokens if usage.prompt_tokens is defined as including the cached tokens (so cached is a subset you subtract back out). That's the OpenAI convention — prompt_tokens is the total and cached_tokens is a subset of it. Anthropic reports it the other way: input_tokens is the non-cached input, with cache_read_input_tokens / cache_creation_input_tokens reported separately and not folded in. If AG2's Usage.prompt_tokens carries the Anthropic-style non-cached value, then for a cached Anthropic call prompt_tokens (say 100) is smaller than cache_read_tokens (say 1000), so cached_input_tokens = min(100, 1000) = 100, regular_input_tokens = 0, and the 100 genuinely-uncached input tokens get billed at zero — an undercharge that the min(...) guard hides rather than surfaces. So the key question: is Usage.prompt_tokens normalized to always include cache tokens across providers before it reaches the estimator? If yes, this is correct; if it's passed through per-provider, the cache subtraction needs to branch on the provider's convention (or the estimator needs the raw non-cached input count).

Two smaller notes:

• cache_read_input_token_cost or pricing.input_cost_per_token (and the cache-creation equivalent) fall back to the full input rate when a cache rate isn't configured. Cache reads are normally much cheaper than fresh input (OpenAI ~0.25-0.5x, Anthropic ~0.1x), so this silently overcharges cache reads rather than flagging that the catalog is missing a cache rate. A conservative default is defensible, but a warning (like the not_configured path already does for a missing model) would keep the estimate honest instead of quietly high.

• The reasoning-token handling (regular_output = completion - thinking only when a reasoning rate is set) assumes completion_tokens includes thinking_tokens, which matches the OpenAI o-series convention — same normalization question as above, just on the output side. Worth one line in the docstring stating that Usage is expected to report completion-inclusive-of-reasoning so a future provider mapping doesn't quietly double count.

The structure is good — it's really just the token-accounting conventions that decide whether the numbers match a real bill, so making the Usage field semantics explicit is what would give me confidence in the totals.