Skip to content

Pulsar: tiered retry-letter error policy MoveToPulsarRetryTopic (#3182)#3207

Merged
jeremydmiller merged 1 commit into
mainfrom
feat/pulsar-retry-dsl-3182
Jun 23, 2026
Merged

Pulsar: tiered retry-letter error policy MoveToPulsarRetryTopic (#3182)#3207
jeremydmiller merged 1 commit into
mainfrom
feat/pulsar-retry-dsl-3182

Conversation

@jeremydmiller

Copy link
Copy Markdown
Member

Closes #3182. Part of the Re-Evaluate Pulsar Integration epic #3176.

Adds a first-class, discoverable retry-letter error policy — the Pulsar analogue of MoveToKafkaRetryTopic:

opts.ListenToPulsarTopic("persistent://public/default/orders")
    .SubscriptionType(SubscriptionType.Shared);

opts.OnException<TransientException>()
    .MoveToPulsarRetryTopic(5.Seconds(), 30.Seconds(), 2.Minutes());

Each TimeSpan is one retry tier: on failure the message is routed through Pulsar's native retry-letter topic with that delay, reprocessed by the retry consumer once it elapses, and after the last tier is exhausted it lands in the dead-letter topic.

How it works

  • MoveToPulsarRetryTopicContinuation (a UserDefinedContinuation, not an opaque CustomAction) self-guards to PulsarListener — a failure arriving over any other transport falls back to a normal inline retry, never cross-transport — and is discoverable in diagnostics.
  • PulsarTransport discovers the policy at startup and wires its per-tier delays onto every Pulsar listener (provisioning the retry-letter producer/consumer + DLQ), so an explicit RetryLetterQueueing(...) is not also required.
  • Startup validation warnings when (a) non-Pulsar listeners are present (policy degrades to inline retry there), or (b) a Pulsar listener uses a subscription type other than Shared/Key_Shared (Pulsar message delaying does not work there).

Acceptance criteria

  • ✅ The DSL routes a failed message through tiered retry-letter delays then to DLQ — integration test (permanent_failure_walks_the_tiers_then_dead_letters: 3 deliveries → 1 dead-lettered).
  • ✅ Applying it to a non-Pulsar listener emits a startup warning — integration test (non_pulsar_listener_emits_a_startup_warning).
  • ✅ Docs section added for Key_Shared by-key concurrency (the free Pulsar analogue of Kafka Kafka: opt-in intra-partition concurrency by key (durable, reuse ShardedExecutionBlock) #3140).

Tests / build

  • pulsar_retry_topic_dsl: 2/2 green against a Testcontainers broker.
  • Full wolverine.slnx builds clean in Release.

🤖 Generated with Claude Code

…yTopic (#3182)

Adds a discoverable error policy DSL mirroring MoveToKafkaRetryTopic:

    opts.OnException<TransientException>()
        .MoveToPulsarRetryTopic(5.Seconds(), 30.Seconds(), 2.Minutes());

Each TimeSpan is a retry tier: on failure the message is routed through Pulsar's
native retry-letter topic with that delay, reprocessed by the retry consumer once
it elapses, and after the last tier exhausts it lands in the dead-letter topic.

- MoveToPulsarRetryTopicContinuation (UserDefinedContinuation) self-guards to
  PulsarListener — failures over any other transport fall back to an inline retry,
  never cross-transport — and is discoverable in diagnostics.
- PulsarTransport wires the policy's per-tier delays onto every Pulsar listener at
  startup (provisioning the retry-letter producer/consumer + DLQ), and emits startup
  warnings when (a) non-Pulsar listeners are present, or (b) a Pulsar listener uses a
  subscription type other than Shared/Key_Shared (where Pulsar message delaying does
  not work and the policy degrades to an inline retry).

Docs: tiered retry-letter policy section + Key_Shared by-key concurrency guidance
(the free Pulsar analogue of Kafka sticky by-key processing).

Tests: permanent failure walks both tiers then dead-letters (3 deliveries -> DLQ);
non-Pulsar listener emits the startup warning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jeremydmiller
jeremydmiller merged commit 4124768 into main Jun 23, 2026
25 checks passed
This was referenced Jun 23, 2026
This was referenced Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pulsar: align retry-letter-topic DSL (MoveToPulsarRetryTopic)

1 participant