Skip to content

Public API to enumerate node-owned databases under managed event subscription distribution (#3340)#3345

Merged
jeremydmiller merged 2 commits into
mainfrom
feat/3340-locally-owned-databases
Jul 8, 2026
Merged

Public API to enumerate node-owned databases under managed event subscription distribution (#3340)#3345
jeremydmiller merged 2 commits into
mainfrom
feat/3340-locally-owned-databases

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Closes #3340.

Problem

Under a MultiTenantedWithShardedDatabases Marten store with
UseWolverineManagedEventSubscriptionDistribution = true, agent assignment is
database-affine (#486/#487): each node only runs the projection/subscription
daemon for the shard databases it owns. But there was no public API for
application code to ask "which databases does the local node own?"

With one physical Postgres database per shard (the reporter runs ~490), a per-pod
component that needs to inspect only its own shards was forced to iterate all
databases — so every pod opened a connection pool to every shard and exhausted a
shared server (53300: remaining connection slots are reserved...). The only
workaround re-implemented the internal EventSubscriptionAgentFamily.DatabaseKeyOf
agent-URI grammar by hand.

Change

  • EventSubscriptionAgentFamily.DatabaseIdOf(Uri) — public, robust extraction
    of the shard DatabaseId from an event-subscriptions:// agent URI. The URI
    grammar stays in the family (mirrors the existing UriFor/BuildAgentAsync
    pair) instead of leaking to callers. Returns null for non-scheme or malformed
    URIs.
  • IAgentRuntime.AllLocallyOwnedDatabaseIds() — the distinct DatabaseId set
    whose event-subscription agents are currently running on this node. Replaces
    the entire fragile workaround with one call.

The reporter's readiness gate becomes:

var owned = runtime.Agents.AllLocallyOwnedDatabaseIds();
foreach (var db in databases.OfType<IMartenDatabase>()
    .Where(d => owned.Contains(d.Identity)))
{
    var progress = await db.AllProjectionProgress(ct); // only this node's shards
}

Tests

  • Fast unit tests for DatabaseIdOf: per-tenant URI, realistic dotted-host sharded
    id round-trip, non-scheme URI, and short-URI null cases.
  • End-to-end assertion added to the existing two-database affine-colocation
    multi-node integration test: each node reports exactly its one owned database,
    matching its running agents, and the two nodes partition the full set.

Both green locally (CoreTests + MartenTests against docker Postgres).

Notes / out of scope

The issue's "bonus" ask — a node-scoped projection-progress query — is store-specific
(Marten) and larger; this PR delivers the small, additive, transport-agnostic core
API that unblocks it. A caller can already build that query by intersecting
AllLocallyOwnedDatabaseIds() with the store's databases as shown above.

🤖 Generated with Claude Code

…subscription distribution (#3340)

Under a MultiTenantedWithShardedDatabases Marten store with
UseWolverineManagedEventSubscriptionDistribution, agent assignment is
database-affine: a node only runs the daemon for the shard databases it owns.
There was no first-class public way for application code (post-deploy
readiness gates, per-node health checks) to discover that same subset, so
per-pod components fanned out a connection pool to every shard database and
exhausted a shared shard server.

Adds:
- EventSubscriptionAgentFamily.DatabaseIdOf(Uri): public, robust extraction of
  the shard DatabaseId from an event-subscriptions:// agent URI. Keeps the URI
  grammar in the family instead of forcing callers to re-implement the internal
  DatabaseKeyOf segment layout.
- IAgentRuntime.AllLocallyOwnedDatabaseIds(): the distinct DatabaseId set whose
  event-subscription agents currently run on THIS node.

Tests: fast unit coverage of DatabaseIdOf (per-tenant URI, realistic dotted-host
sharded id round-trip, non-scheme and short-URI null cases) plus an end-to-end
assertion in the two-database affine-colocation multi-node test that each node
reports exactly its owned database and the two nodes partition the set.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jeremydmiller
jeremydmiller merged commit 2285475 into main Jul 8, 2026
25 checks passed
This was referenced Jul 9, 2026
This was referenced Jul 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Public API to enumerate the databases owned by the local node under managed event-subscription distribution

1 participant