columnar: gc ia meta & schema files in cloud_helper

[yongman]

What problem does this PR solve?

Issue Number: close #10995

Problem Summary:
The disk or memory can be keep increasing in the long run with no background gc.

What is changed and how it works?

1. Add gc for ia_meta files in disk. (In TiKV there has a background gc task to scan the local meta files and do the gc)
2. Add gc for schema_files map in memory.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

Summary by CodeRabbit

• New Features

  • Added a configuration option to cap the number of cached schema keyspaces.
  • Enabled conditional background cleanup for schema-file cache and IA metadata, controlled by the local GC interval.

• Bug Fixes

  • Improved cleanup correctness by refreshing cache recency on snapshot access.
  • Refined shutdown coordination so cleanup and heartbeat background tasks stop reliably.

• Tests

  • Added/extended tests for stale schema-version pruning, recency/LRU ordering, and GC eviction with and without keyspace limits.
  • Updated configuration parsing assertions for the new local GC defaults.

[ti-chi-bot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign zanmato1984 for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

📝 Walkthrough

Walkthrough

Schema-file cache recency tracking and global eviction are added to CloudHelper. Runtime configuration now controls IA metadata and schema cache GC, with a background thread, coordinated shutdown, and updated defaults.

Changes

Runtime garbage collection

Layer / File(s)	Summary
Schema cache tracking and eviction contrib/tiflash-columnar-hub/hub-runtime/src/cloud_helper.rs	Adds per-keyspace cache state, recency updates, stale-version pruning, configurable keyspace eviction, and unit tests.
Snapshot access recency integration contrib/tiflash-columnar-hub/hub-runtime/src/cloud_helper.rs	Propagates shared cache state through snapshot loading and touches the active keyspace after snapshot creation.
Configurable background GC lifecycle contrib/tiflash-columnar-hub/hub-runtime/src/run.rs	Adds GC configuration, starts periodic IA metadata and schema cache cleanup, coordinates shutdown, and joins the background thread.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant EngineStoreLifecycle
  participant BackgroundGCLoop
  participant CloudHelper
  participant IAMetaFiles
  EngineStoreLifecycle->>BackgroundGCLoop: start when local_gc.interval is non-zero
  BackgroundGCLoop->>IAMetaFiles: run IA metadata GC
  BackgroundGCLoop->>CloudHelper: gc_schema_file_cache()
  CloudHelper->>CloudHelper: prune and evict schema-file entries
  EngineStoreLifecycle->>BackgroundGCLoop: set service_shutdown
  BackgroundGCLoop-->>EngineStoreLifecycle: exit and join

Loading

Suggested reviewers: lloyd-pottiger, jayson-huang

Poem

I’m a rabbit with cache in my burrow tonight,
Touching warm keyspaces, pruning old bytes from sight.
IA files hop cleanly, the GC loops with care,
Shutdown bells ring softly through the runtime air.
~( ,_,_,/)
( •ㅅ• )

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The template is mostly followed, but the checklist lacks any marked tests and the side effects/documentation fields are left empty.	Add at least one checked test item, and fill in any relevant side effects or documentation notes so the template is complete.
Docstring Coverage	⚠️ Warning	Docstring coverage is 22.58% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: garbage collection for ia meta and schema files in the columnar hub.
Linked Issues check	✅ Passed	The changes implement background GC for both ia_meta files on disk and schema_files in memory as required by #10995.
Out of Scope Changes check	✅ Passed	No clearly unrelated changes are present; the config and shutdown refactors support the new GC behavior.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@contrib/tiflash-columnar-hub/hub-runtime/src/cloud_helper.rs`:
- Around line 1390-1399: Update the cache-state insertion in the
latest_by_keyspace loop to capture the current access_seq and assign it to
last_access_seq for newly discovered keyspaces instead of 0. Keep existing
entries’ latest_version updates unchanged, so freshly loaded keyspaces are
treated as recently accessed during gc_schema_file_cache_global eviction.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 9faf5786-67f3-4e03-b576-cc762b795186

📥 Commits

Reviewing files that changed from the base of the PR and between 3d986e3 and f7f5c20.

📒 Files selected for processing (2)

• contrib/tiflash-columnar-hub/hub-runtime/src/cloud_helper.rs
• contrib/tiflash-columnar-hub/hub-runtime/src/run.rs

[yongman]

/test pull-integration-test

[yongman]

/hold

[coderabbitai]

🧹 Nitpick comments (1)

contrib/tiflash-columnar-hub/hub-runtime/src/run.rs (1)

1096-1119: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift

Add an end-to-end GC lifecycle test.

The added test only validates TOML parsing; it does not prove the spawned loop removes expired IA metadata, evicts schema-cache entries, or exits promptly on shutdown. Add a deterministic integration test using a temporary data directory and a short interval.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@contrib/tiflash-columnar-hub/hub-runtime/src/run.rs` around lines 1096 -
1119, Add a deterministic end-to-end test for spawn_background_gc_loop using a
temporary data directory and short GC interval. Exercise expired IA metadata
cleanup and schema-cache eviction, then signal shutdown and join the returned
thread, asserting the loop exits promptly and both cleanup effects occur rather
than only validating TOML parsing.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@contrib/tiflash-columnar-hub/hub-runtime/src/run.rs`:
- Around line 1096-1119: Add a deterministic end-to-end test for
spawn_background_gc_loop using a temporary data directory and short GC interval.
Exercise expired IA metadata cleanup and schema-cache eviction, then signal
shutdown and join the returned thread, asserting the loop exits promptly and
both cleanup effects occur rather than only validating TOML parsing.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 693b2191-cd14-42c7-9df9-92a185f8b26c

📥 Commits

Reviewing files that changed from the base of the PR and between 97f3f53 and d39fe33.

📒 Files selected for processing (2)

• contrib/tiflash-columnar-hub/hub-runtime/src/cloud_helper.rs
• contrib/tiflash-columnar-hub/hub-runtime/src/run.rs

🚧 Files skipped from review as they are similar to previous changes (1)

• contrib/tiflash-columnar-hub/hub-runtime/src/cloud_helper.rs

[ti-chi-bot]

@yongman: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-sanitizer-tsan	d39fe33	link	false	/test pull-sanitizer-tsan
pull-sanitizer-asan	d39fe33	link	false	/test pull-sanitizer-asan

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.