Skip to content

feat(meta-service): Add comprehensive snapshot database metrics - #18422

Merged
drmingdrmer merged 1 commit into
databendlabs:mainfrom
drmingdrmer:361-snapshot-metrics
Jul 25, 2025
Merged

feat(meta-service): Add comprehensive snapshot database metrics#18422
drmingdrmer merged 1 commit into
databendlabs:mainfrom
drmingdrmer:361-snapshot-metrics

Conversation

@drmingdrmer

@drmingdrmer drmingdrmer commented Jul 25, 2025

Copy link
Copy Markdown
Member

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

feat(meta-service): Add comprehensive snapshot database metrics

Add detailed snapshot database metrics to monitor storage performance and statistics:

  • Block count, data size, index size metrics
  • Average block size and keys per block calculations
  • Block read statistics (total, from cache, from disk)
  • Upgrade rotbl dependency to 0.2.7 for enhanced access statistics

These metrics provide visibility into snapshot storage efficiency and cache performance.

All metrics are gauge type:

# number of keys in the last snapshot.
metasrv_server_snapshot_key_count 0

# number of primary keys in the last snapshot.
metasrv_server_snapshot_primary_index_count 0

# number of expire index keys in the last snapshot.
metasrv_server_snapshot_expire_index_count 0

# number of blocks in the last snapshot.
metasrv_server_snapshot_block_count 0

# size of data section in the last snapshot.
metasrv_server_snapshot_data_size 0

# size of index section in the last snapshot.
metasrv_server_snapshot_index_size 0

# average size of a block in the last snapshot.
metasrv_server_snapshot_avg_block_size 0

# average number of keys per block in the last snapshot.
metasrv_server_snapshot_avg_keys_per_block 0

# total number of read block from cache or from disk.
metasrv_server_snapshot_read_block 0

# total number of read block from cache.
metasrv_server_snapshot_read_block_from_cache 0

# total number of read block from disk.
metasrv_server_snapshot_read_block_from_disk 0

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Other

Related Issues


This change is Reviewable

@github-actions github-actions Bot added the pr-feature this PR introduces a new feature to the codebase label Jul 25, 2025
@drmingdrmer
drmingdrmer force-pushed the 361-snapshot-metrics branch from 3d70472 to bc964fc Compare July 25, 2025 06:19
@drmingdrmer
drmingdrmer marked this pull request as ready for review July 25, 2025 06:19
@drmingdrmer
drmingdrmer requested a review from everpcpc July 25, 2025 06:19
Add detailed snapshot database metrics to monitor storage performance and statistics:
- Block count, data size, index size metrics
- Average block size and keys per block calculations
- Block read statistics (total, from cache, from disk)
- Upgrade rotbl dependency to 0.2.7 for enhanced access statistics

These metrics provide visibility into snapshot storage efficiency and cache performance.

All metrics are `gauge` type:

```config
# number of keys in the last snapshot.
metasrv_server_snapshot_key_count 0

# number of primary keys in the last snapshot.
metasrv_server_snapshot_primary_index_count 0

# number of expire index keys in the last snapshot.
metasrv_server_snapshot_expire_index_count 0

# number of blocks in the last snapshot.
metasrv_server_snapshot_block_count 0

# size of data section in the last snapshot.
metasrv_server_snapshot_data_size 0

# size of index section in the last snapshot.
metasrv_server_snapshot_index_size 0

# average size of a block in the last snapshot.
metasrv_server_snapshot_avg_block_size 0

# average number of keys per block in the last snapshot.
metasrv_server_snapshot_avg_keys_per_block 0

# total number of read block from cache or from disk.
metasrv_server_snapshot_read_block 0

# total number of read block from cache.
metasrv_server_snapshot_read_block_from_cache 0

# total number of read block from disk.
metasrv_server_snapshot_read_block_from_disk 0
```
@drmingdrmer
drmingdrmer force-pushed the 361-snapshot-metrics branch from bc964fc to b295464 Compare July 25, 2025 08:31
@drmingdrmer
drmingdrmer merged commit 2c99447 into databendlabs:main Jul 25, 2025
@drmingdrmer
drmingdrmer deleted the 361-snapshot-metrics branch July 25, 2025 09:29
TCeason pushed a commit to TCeason/datafuse that referenced this pull request Jul 25, 2025
…bendlabs#18422)

Add detailed snapshot database metrics to monitor storage performance and statistics:
- Block count, data size, index size metrics
- Average block size and keys per block calculations
- Block read statistics (total, from cache, from disk)
- Upgrade rotbl dependency to 0.2.7 for enhanced access statistics

These metrics provide visibility into snapshot storage efficiency and cache performance.

All metrics are `gauge` type:

```config
metasrv_server_snapshot_key_count 0

metasrv_server_snapshot_primary_index_count 0

metasrv_server_snapshot_expire_index_count 0

metasrv_server_snapshot_block_count 0

metasrv_server_snapshot_data_size 0

metasrv_server_snapshot_index_size 0

metasrv_server_snapshot_avg_block_size 0

metasrv_server_snapshot_avg_keys_per_block 0

metasrv_server_snapshot_read_block 0

metasrv_server_snapshot_read_block_from_cache 0

metasrv_server_snapshot_read_block_from_disk 0
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants