Skip to content

feat(meta): add KvList gRPC API for listing keys by prefix - #19235

Merged
drmingdrmer merged 3 commits into
databendlabs:mainfrom
drmingdrmer:345-grpc-list
Jan 12, 2026
Merged

feat(meta): add KvList gRPC API for listing keys by prefix#19235
drmingdrmer merged 3 commits into
databendlabs:mainfrom
drmingdrmer:345-grpc-list

Conversation

@drmingdrmer

@drmingdrmer drmingdrmer commented Jan 12, 2026

Copy link
Copy Markdown
Member

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

feat(meta): add KvList gRPC API for listing keys by prefix

Add streaming KvList RPC that lists key-value pairs matching a prefix.
Requires leadership - non-leaders return error with leader endpoint in metadata.

Changes:

  • Add KvList RPC to MetaService proto with KvListRequest message
  • Add kv_list() method to MetaLeader for listing keys from state machine
  • Add handle_kv_list() to MetaNode with leader check and redirect
  • Add status_forward_to_leader() to GrpcHelper for error responses
  • Refactor GrpcHelper to share metadata parsing logic
  • Add kv_list() to EstablishedClient
  • Add integration tests for leader, follower redirect, and no-quorum scenarios
docs: add examples and sample output to metactl subcommand help

Improve the CLI help messages for all metactl subcommands by adding
usage examples and sample output. This helps users understand how to
use each command without referring to external documentation.

Changes:

  • Add description, example usage, and sample output for all 16 subcommands
  • Use #[command(verbatim_doc_comment)] to preserve formatting in help text
  • Include multiple examples where commands have different usage patterns

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Other

Related Issues


This change is Reviewable

@github-actions github-actions Bot added the pr-feature this PR introduces a new feature to the codebase label Jan 12, 2026
@drmingdrmer
drmingdrmer force-pushed the 345-grpc-list branch 2 times, most recently from 006f015 to 8af4b17 Compare January 12, 2026 06:50

@xp-trumpet xp-trumpet left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xp-trumpet reviewed 12 files and all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @drmingdrmer).

@xp-trumpet xp-trumpet left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xp-trumpet reviewed 1 file and all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @drmingdrmer).

Improve the CLI help messages for all `metactl` subcommands by adding
usage examples and sample output. This helps users understand how to
use each command without referring to external documentation.

Changes:
- Add description, example usage, and sample output for all 16 subcommands
- Use `#[command(verbatim_doc_comment)]` to preserve formatting in help text
- Include multiple examples where commands have different usage patterns
Add streaming `KvList` RPC that lists key-value pairs matching a prefix.
Requires leadership - non-leaders return error with leader endpoint in metadata.

Changes:
- Add `KvList` RPC to `MetaService` proto with `KvListRequest` message
- Add `kv_list()` method to `MetaLeader` for listing keys from state machine
- Add `handle_kv_list()` to `MetaNode` with leader check and redirect
- Add `status_forward_to_leader()` to `GrpcHelper` for error responses
- Refactor `GrpcHelper` to share metadata parsing logic
- Add `kv_list()` to `EstablishedClient`
- Add integration tests for leader, follower redirect, and no-quorum scenarios
…ernal_resets` error

The h2 library has a default limit of 1024 locally-reset streams per
connection. When this limit is reached, h2 sends a GoAway frame with
ENHANCE_YOUR_CALM error and `too_many_internal_resets` message. This
can happen under high concurrency when streams are cancelled or dropped
before completion.

This fix adds a connection TTL (default 20 seconds) to force periodic
reconnection, preventing stream reset accumulation on long-lived connections.

Changes:
- Add `created_instant` field to `EstablishedClient` for TTL tracking
- Add `is_expired()` method to check connection age
- Add `connection_ttl` field to `MetaChannelManager` (default: 20s)
- Check TTL in `MetaChannelManager::check()` and reject expired connections
- Configure server-side `http2_max_pending_accept_reset_streams(4096)` as safety measure
@drmingdrmer
drmingdrmer merged commit 5757d76 into databendlabs:main Jan 12, 2026
88 of 89 checks passed
@drmingdrmer
drmingdrmer deleted the 345-grpc-list branch January 12, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants