Skip to content

fix: BeginAtEarliest/BeginAtLatest drops GroupId from transport-level ConsumerConfig#3174

Merged
jeremydmiller merged 1 commit into
JasperFx:mainfrom
lahma:fix/begin-at-earliest-group-id-inheritance
Jun 22, 2026
Merged

fix: BeginAtEarliest/BeginAtLatest drops GroupId from transport-level ConsumerConfig#3174
jeremydmiller merged 1 commit into
JasperFx:mainfrom
lahma:fix/begin-at-earliest-group-id-inheritance

Conversation

@lahma

@lahma lahma commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Bug

KafkaListenerConfiguration.BeginAtEarliest() / BeginAtLatest() call:

topic.ConsumerConfig ??= new ConsumerConfig();
topic.ConsumerConfig.AutoOffsetReset = AutoOffsetReset.Earliest;

This creates a fresh per-topic ConsumerConfig with only AutoOffsetReset set.
KafkaTopic.GetEffectiveConsumerConfig() then returns this sparse config — it already
copied BootstrapServers from the parent, but not GroupId. Confluent.Kafka requires
group.id and throws at startup:

System.ArgumentException: 'group.id' configuration parameter is required and was not specified.

This is a breaking change introduced with the BeginAtEarliest() / BeginAtLatest()
methods added in 6.13.x (GH-3146).

Root cause

GetEffectiveConsumerConfig() in KafkaTopic.cs:

internal ConsumerConfig GetEffectiveConsumerConfig()
{
    if (ConsumerConfig != null && string.IsNullOrEmpty(ConsumerConfig.BootstrapServers))
    {
        ConsumerConfig.BootstrapServers = Parent.ConsumerConfig.BootstrapServers;
    }
    // GroupId was never propagated — the returned config has no group.id
    return ConsumerConfig ?? Parent.ConsumerConfig;
}

Fix

Add GroupId inheritance immediately after BootstrapServers, using the same pattern:

if (ConsumerConfig != null && string.IsNullOrEmpty(ConsumerConfig.GroupId))
{
    ConsumerConfig.GroupId = Parent.ConsumerConfig.GroupId;
}

Tests

Three new unit tests in KafkaListenerConfigurationTests:

  • begin_at_earliest_inherits_group_id_from_parent_transport — verifies GroupId + AutoOffsetReset.Earliest + BootstrapServers are all present after .BeginAtEarliest()
  • begin_at_latest_inherits_group_id_from_parent_transport — same for .BeginAtLatest()
  • begin_at_earliest_does_not_override_explicitly_set_group_id — verifies a per-topic GroupId set via ConfigureConsumer is not overwritten

Workaround (for users who cannot wait for a fix release)

Use ExtendConsumerConfiguration instead of BeginAtEarliest():

opts.ListenToKafkaTopic("my-topic")
    .ExtendConsumerConfiguration(c => c.AutoOffsetReset = AutoOffsetReset.Earliest);

… ConsumerConfig

GetEffectiveConsumerConfig() already inherited BootstrapServers from the parent
transport config, but not GroupId. When BeginAtEarliest() or BeginAtLatest() was
called on a per-listener KafkaListenerConfiguration, it created a fresh per-topic
ConsumerConfig with only AutoOffsetReset set. GetEffectiveConsumerConfig() would
then return that sparse config, which has no GroupId, causing Confluent.Kafka to
throw "group.id configuration parameter is required" at startup.

Add GroupId inheritance in GetEffectiveConsumerConfig() using the same pattern
already used for BootstrapServers. Add three unit tests covering the fix.
Copilot AI review requested due to automatic review settings June 22, 2026 10:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a regression in the Kafka transport where calling KafkaListenerConfiguration.BeginAtEarliest() / BeginAtLatest() could result in a per-topic ConsumerConfig missing GroupId, causing Confluent.Kafka startup failures (group.id is required).

Changes:

  • Update KafkaTopic.GetEffectiveConsumerConfig() to inherit GroupId from the parent transport when a topic-level ConsumerConfig exists but has no GroupId.
  • Add unit tests covering BeginAtEarliest() / BeginAtLatest() inheritance of GroupId and ensuring an explicitly configured per-topic GroupId is not overridden.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/Transports/Kafka/Wolverine.Kafka/KafkaTopic.cs Ensures topic-level consumer configs inherit GroupId from the parent transport when missing.
src/Transports/Kafka/Wolverine.Kafka.Tests/KafkaTransportTests.cs Adds regression tests validating BeginAtEarliest/Latest behavior and GroupId inheritance semantics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jeremydmiller
jeremydmiller merged commit b21a6d8 into JasperFx:main Jun 22, 2026
24 checks passed
This was referenced Jun 23, 2026
This was referenced Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants