Skip to content

CosmosDB saga persistence has no optimistic concurrency — a blind UpsertItemAsync loses updates under concurrent messages #3414

Description

@jeremydmiller

Found reviewing #3410 (which adds F# codegen for these frames). This is a pre-existing C# defect that the F# work faithfully inherits — not a regression from that PR.

The gap

Saga.Version exists (src/Wolverine/Saga.cs:39), and Wolverine.Marten's LoadDocumentFrame honors it — it maps the saga through ExpectedSagaRevision / IRevisioned so a concurrent write is detected and retried.

The CosmosDB provider does not. CosmosDbPersistenceFrameProvider persists with a blind UpsertItemAsync(saga) — no etag captured on the point read, no ItemRequestOptions.IfMatchEtag on the write.

Failure scenario

Two messages for the same saga id land on two nodes (or two threads on one node) at the same time:

  1. Node A point-reads ThingSaga { Count = 1 }.
  2. Node B point-reads ThingSaga { Count = 1 }.
  3. Node A applies its message → Count = 2UpsertItemAsync → stored.
  4. Node B applies its message → Count = 2UpsertItemAsyncoverwrites.

Final state: Count = 2. Correct: Count = 3. Lost update, silently. No exception, no retry, no dead letter — the saga just quietly has the wrong state, which for a saga (a state machine coordinating a long-running workflow) is about the worst possible failure mode.

Cosmos gives us exactly the primitive needed here: every item read carries an ETag, and ItemRequestOptions.IfMatchEtag turns the upsert into a compare-and-swap that 412s on conflict. That 412 should surface the same way Marten's concurrency exception does, so Wolverine's existing saga-retry machinery handles it.

Ask

Capture the ETag on the saga point read, pass it as IfMatchEtag on the upsert/delete, and map the resulting 412 onto whatever exception the saga retry path already understands (see how Marten's LoadDocumentFrame wires ExpectedSagaRevision). Needs a concurrency test in CosmosDbTests — two concurrent messages against one saga id, assert both applied.

Refs #3410.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions