merge: from 3.3.6 to 3.3.8

[wangmm0220]

Description

Issue(s)

• Close/close/Fix/fix/Resolve/resolve: Issue Link

Checklist

Please check the items in the checklist if applicable.

• Is the user manual updated?
• Are the test cases passed and automated?
• Is there no significant decrease in test coverage?

[gemini-code-assist]

Summary of Changes

Hello @wangmm0220, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors and enhances the Time-series Message Queue (TMQ) system's ability to manage and filter tables based on tag conditions. It introduces a more modular approach to handling table list updates, separating add, delete, and comprehensive update operations. A key improvement is the robust integration with ALTER TABLE ... SET TAG commands, ensuring that TMQ consumers dynamically adapt their subscriptions to reflect changes in table tags. This is crucial for maintaining data consistency and accuracy in streaming analytics when metadata evolves. The changes are thoroughly validated with new unit and system tests, including scenarios involving frequent tag alterations on a large scale.

Highlights

• Refactored TMQ Table List Management: The core functions for updating Time-series Message Queue (TMQ) table lists, qUpdateTableListForStreamScanner and tqUpdateTbUidList, have been refactored into distinct add, delete, and update operations. This provides more granular control and clarity over how table lists are managed for stream scanners and TMQ consumers.
• Enhanced Tag-Based Filtering for TMQ: New functions such as checkCidInTagCondition, getTagCondNodeForStableTmq, and getTagCondNodeForQueryTmq have been introduced. These functions enable more precise evaluation and application of tag conditions during table list filtering, ensuring that TMQ subscriptions accurately reflect metadata changes.
• Improved ALTER TABLE ... SET TAG Integration: The ALTER TABLE ... SET TAG operation now seamlessly integrates with TMQ. A new alterTagForTmq function ensures that when table tags are modified, TMQ subscriptions are dynamically updated, allowing consumers to receive data based on the latest tag values.
• Comprehensive New System Tests: Two new Python system tests, test_tmq_update_tablelist.py and tmq_alter_tag_frequency.py, have been added. These tests validate the correctness and performance of TMQ filtering under various tag alteration scenarios, including high-frequency updates on a large number of tables, ensuring robustness and reliability.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Copilot]

Pull request overview

This pull request merges changes from version 3.3.6 to 3.3.8, focusing on improving TMQ (TDengine Message Queue) table list management when tags are altered. The changes refactor the table update mechanism to better handle scenarios where tag values change, which affects topic subscriptions with tag-based filtering.

Changes:

• Refactored TMQ table list management to split operations into separate add, update, and delete functions
• Added new test cases for TMQ behavior when tags are frequently altered
• Optimized tag filtering logic to avoid unnecessary reprocessing and improve performance

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 21 comments.

Show a summary per file

File	Description
tests/system-test/7-tmq/tmq_alter_tag_frequency.py	New test for TMQ behavior with frequent tag alterations
test/cases/17-DataSubscription/02-Consume/test_tmq_update_tablelist.py	New test for table list updates in TMQ subscriptions
tests/parallel_test/cases.task	Added new test case to parallel test suite
test/ci/cases.task	Added new test case to CI pipeline
source/libs/executor/src/executor.c	Refactored table list management - split single function into add/delete/update operations
source/libs/executor/src/executil.c	Refactored tag filtering logic with optimizations for tbname conditions
source/libs/executor/inc/executil.h	Updated function signatures for tag filtering
source/dnode/vnode/src/vnd/vnodeSvr.c	Updated function calls to use new split API and added alterTagForTmq helper
source/dnode/vnode/src/tq/tqRead.c	Major refactoring - split tqUpdateTbUidList into three functions
source/dnode/vnode/src/tq/tqPush.c	Changed log level from debug to info
source/dnode/vnode/src/meta/metaQuery.c	Added optimization to skip already-fetched tag values
source/dnode/vnode/src/inc/vnodeInt.h	Updated function signatures for new split API
source/client/src/clientTmq.c	Added periodic logging for vgroup skip counts
include/libs/executor/executor.h	Public API updates for table list management

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[gemini-code-assist]

Code Review

This pull request introduces a significant refactoring of the TMQ table list update mechanism, primarily to support dynamic updates when table tags are altered. The changes are well-structured, separating the logic for adding, deleting, and updating table lists into distinct functions, which improves clarity and maintainability. The implementation for ALTER TAG handling includes optimizations to only re-evaluate tables when necessary. I've also noticed and appreciate the inclusion of important memory leak fixes and new test cases for this new functionality.

I have a few minor suggestions for improvement regarding an outdated comment, use of a magic number, and a note on error handling logic that could lead to inconsistencies. Overall, this is a solid set of changes for a complex feature.