Add processed_mentions tracking to prevent duplicate replies and improve fallback message - #110
Add processed_mentions tracking to prevent duplicate replies and improve fallback message#110groupthinking wants to merge 1 commit into
Conversation
…ove fallback message
📝 WalkthroughSummary
Walkthrough
ChangesMention processing
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant X_API
participant listener.py
participant Grok
participant processed_mentions_file
X_API->>listener.py: provide mentions
listener.py->>listener.py: skip previously processed IDs
listener.py->>Grok: request reply
Grok-->>listener.py: reply or failure
listener.py->>X_API: create reply tweet
listener.py->>processed_mentions_file: persist mention ID
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 3❌ Failed checks (2 warnings, 1 inconclusive)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
✨ Simplify code
Warning Review ran into problems🔥 ProblemsThese MCP integrations need to be re-authenticated in the Integrations settings: Sentry Comment |
There was a problem hiding this comment.
Pull request overview
Adds persistent tracking of processed X mention IDs in the Python listener to prevent duplicate replies across polling cycles/restarts, and updates the fallback reply message when Grok generation fails.
Changes:
- Persist processed mention IDs to a local file and skip already-processed mentions.
- Update the Grok error fallback reply text.
- Change the listener’s default Timeline API URL (currently to port 8000).
Comments suppressed due to low confidence (1)
listener.py:100
- The default TIMELINE_API_URL port was changed to 8000 here, but the rest of the repo (env.example, docker-compose, mcp_dispatcher.py) still defaults the timeline server to 8080. If TIMELINE_API_URL isn’t set, agent registration will POST to the wrong local endpoint.
timeline_url = os.getenv("TIMELINE_API_URL", "http://127.0.0.1:8000")
|
|
||
| def push_timeline_card(title: str, body: str, metadata: dict) -> None: | ||
| timeline_url = os.getenv("TIMELINE_API_URL", "http://127.0.0.1:8080") | ||
| timeline_url = os.getenv("TIMELINE_API_URL", "http://127.0.0.1:8000") |
| mention_id_str = str(mention.id) | ||
| if mention_id_str in processed_mentions: | ||
| print(f"Skipping already processed mention {mention.id}", flush=True) | ||
| continue | ||
|
|
| try: | ||
| client.create_tweet( | ||
| text=grok_reply[:280], | ||
| in_reply_to_tweet_id=mention.id, | ||
| ) | ||
| processed_mentions.add(mention_id_str) | ||
| save_processed_mention(mention_id_str) | ||
| except Exception as exc: | ||
| print(f"Error replying to mention {mention.id}: {exc}", flush=True) |
| def load_processed_mentions() -> set[str]: | ||
| if not PROCESSED_MENTIONS_PATH.exists(): | ||
| return set() | ||
| with PROCESSED_MENTIONS_PATH.open("r", encoding="utf-8") as f: | ||
| return {line.strip() for line in f if line.strip()} |
| POLL_SECONDS = int(os.getenv("POLL_INTERVAL_SECONDS", "60")) | ||
| PAYMENT_REQUIRED_BACKOFF_SECONDS = int(os.getenv("X_PAYMENT_REQUIRED_BACKOFF_SECONDS", "900")) | ||
|
|
||
| PROCESSED_MENTIONS_PATH = Path(os.getenv("XMCP_PROCESSED_MENTIONS_PATH", "~/.xmcp/processed_mentions.txt")).expanduser() |
![coderabbitai[bot]](https://avatars.github.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@listener.py`:
- Around line 45-48: Replace the unbounded append-only storage in
save_processed_mention with a bounded durable store, or add safe
compaction/retention tied to the listener’s replay window. Ensure compaction
preserves every mention ID still needed to prevent replay while removing older
entries, and keep startup loading bounded accordingly.
- Around line 171-174: Update the get_grok_reply flow in the mention handler so
failure-like return values such as “Missing XAI_API_KEY.” and “Thinking...” are
detected before create_tweet(). Route them through the existing apology fallback
and ensure they are not published or recorded as successful replies; preserve
the current exception handling behavior for raised errors.
- Around line 38-42: Update load_processed_mentions() to handle filesystem read
errors without silently returning an empty set: retry transient failures or
propagate an explicit health failure. Adjust main() so failures from
load_processed_mentions() are handled within the daemon polling lifecycle,
preventing the listener thread from terminating while preserving the fail-closed
behavior that avoids duplicate replies.
- Around line 176-182: The mention handler around create_tweet and
save_processed_mention must distinguish successful tweet delivery from durable
state persistence. Do not let save_processed_mention failures enter the “Error
replying” path or treat the tweet as unsent; make persistence failures retry or
fail closed, and only update processed_mentions consistently with a successful
durable commit to prevent replay after restart.
- Line 86: Align the TIMELINE_API_URL fallback consistently across listener.py,
env.example, and agents/base.py, using the same endpoint default everywhere
(preferably port 8080 to match the existing configuration). Update the listener
registration and timeline-card request paths and the documented example
together, or make the variable mandatory in all locations.
- Around line 156-159: Update the mention-processing flow around the
processed_mentions check so the persisted start_time/cursor advances
monotonically to the newest processed mention, rather than being overwritten for
each newest-first item. Track the maximum tweet ID or timestamp across handled
mentions and persist that value only after processing the batch, while
preserving the skip behavior for already processed mentions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 4e25f4c6-1faa-49f0-b6cb-fd85c8c5c777
📒 Files selected for processing (1)
listener.py
🔗 Linked repositories identified
CodeRabbit considers these linked repositories for cross-repo context during reviews:
groupthinking/uvai-skills(manual)
📜 Review details
⏰ Context from checks skipped due to timeout. (1)
- GitHub Check: copilot-pull-request-reviewer
🧰 Additional context used
🔍 Remote MCP GitHub Copilot
Additional PR context
- PR
#110is open, has 1 commit, 1 changed file (listener.py), with +26/-4 lines; mergeable state isdirty. An auto-generated CodeRabbit comment says review is still in progress. main’slistener.pyalready persistslast_seenand processes mentions oldest-first; this PR adds a separate persistedprocessed_mentionsset viaXMCP_PROCESSED_MENTIONS_PATH, loads it at startup, skips already-processed mention IDs, and saves IDs after a successful reply.- The Grok-failure fallback text changes from
Processing your tag... (error generating full response)toSorry, I'm having trouble processing that. Try again or DM me. - The timeline API default in
listener.pychanges tohttp://127.0.0.1:8000, butenv.exampleandagents/base.pystill defaultTIMELINE_API_URLtohttp://127.0.0.1:8080;env.examplealso setsMCP_PORT=8000andMCP_SERVER_URL=http://127.0.0.1:8000/mcp.
🔇 Additional comments (1)
listener.py (1)
19-19: LGTM!Also applies to: 129-130, 140-141
| def load_processed_mentions() -> set[str]: | ||
| if not PROCESSED_MENTIONS_PATH.exists(): | ||
| return set() | ||
| with PROCESSED_MENTIONS_PATH.open("r", encoding="utf-8") as f: | ||
| return {line.strip() for line in f if line.strip()} |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Do not let a broken state file kill the listener.
load_processed_mentions() lets filesystem errors escape, and main() calls it outside the polling error handler. Because main.py:49-58 runs this in a daemon thread, an unreadable or temporarily unavailable file terminates mention processing. Retry or fail closed with an explicit health failure; never silently use an empty set, since that re-enables duplicate replies.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@listener.py` around lines 38 - 42, Update load_processed_mentions() to handle
filesystem read errors without silently returning an empty set: retry transient
failures or propagate an explicit health failure. Adjust main() so failures from
load_processed_mentions() are handled within the daemon polling lifecycle,
preventing the listener thread from terminating while preserving the fail-closed
behavior that avoids duplicate replies.
| def save_processed_mention(mention_id: str) -> None: | ||
| PROCESSED_MENTIONS_PATH.parent.mkdir(parents=True, exist_ok=True) | ||
| with PROCESSED_MENTIONS_PATH.open("a", encoding="utf-8") as f: | ||
| f.write(f"{mention_id}\n") |
There was a problem hiding this comment.
🚀 Performance & Scalability | 🟠 Major | 🏗️ Heavy lift
This processed-mention ledger grows forever.
Every successful mention appends another line, while startup loads the entire file into memory. Long-running usage will cause unbounded disk growth and increasingly expensive restarts. Use a bounded durable store or implement safe compaction/retention based on the replay window.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@listener.py` around lines 45 - 48, Replace the unbounded append-only storage
in save_processed_mention with a bounded durable store, or add safe
compaction/retention tied to the listener’s replay window. Ensure compaction
preserves every mention ID still needed to prevent replay while removing older
entries, and keep startup loading bounded accordingly.
|
|
||
| def push_timeline_card(title: str, body: str, metadata: dict) -> None: | ||
| timeline_url = os.getenv("TIMELINE_API_URL", "http://127.0.0.1:8080") | ||
| timeline_url = os.getenv("TIMELINE_API_URL", "http://127.0.0.1:8000") |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Align the timeline endpoint default everywhere.
listener.py now defaults to port 8000, while env.example and agents/base.py still default 8080. With no explicit TIMELINE_API_URL, registration and timeline-card requests hit the wrong service. Update all defaults together or require the variable explicitly.
Also applies to: 100-100
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@listener.py` at line 86, Align the TIMELINE_API_URL fallback consistently
across listener.py, env.example, and agents/base.py, using the same endpoint
default everywhere (preferably port 8080 to match the existing configuration).
Update the listener registration and timeline-card request paths and the
documented example together, or make the variable mandatory in all locations.
| mention_id_str = str(mention.id) | ||
| if mention_id_str in processed_mentions: | ||
| print(f"Skipping already processed mention {mention.id}", flush=True) | ||
| continue |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -eu
rg -n 'get_users_mentions|start_time|since_id|last_seen|processed_mentions' listener.pyRepository: groupthinking/MyXstack
Length of output: 1107
🏁 Script executed:
#!/bin/bash
set -eu
sed -n '110,215p' listener.py | cat -nRepository: groupthinking/MyXstack
Length of output: 4377
🌐 Web query:
Tweepy get_users_mentions start_time inclusive second granularity order returned mentions newest first docs
💡 Result:
In Tweepy, the get_users_mentions method uses the Twitter API v2, which processes time parameters with second granularity [1][2]. Regarding your specific questions: 1. Inclusive Granularity: The start_time and end_time parameters are inclusive [1][2]. For example, a timestamp set to 12:00:01 includes the first second of that minute [1][2]. 2. Order of Returns: The endpoint returns mentions in reverse chronological order (newest first) by default. 3. Documentation: These details are documented in the Tweepy official documentation for Client.get_users_mentions [1][3]. Note that this endpoint is subject to Twitter API v2 limitations, such as restricted history access (typically up to the last 800 mentions) [1][4]. If you need access to older data, you may need to utilize different API endpoints depending on your access level [4].
Citations:
- 1: https://docs.tweepy.org/en/v4.14.0/client.html
- 2: https://docs.tweepy.org/en/v4.10.1/asyncclient.html
- 3: https://docs.tweepy.org/en/stable/client.html
- 4: https://stackoverflow.com/questions/72170210/tweepy-not-retrieving-older-mentions
🏁 Script executed:
#!/bin/bash
set -eu
sed -n '1,110p' listener.py | cat -nRepository: groupthinking/MyXstack
Length of output: 4603
Checkpoint is moving the wrong way
start_time gets overwritten for every mention, and Tweepy returns mentions newest-first. That leaves the next poll anchored to the oldest item already handled, so you keep re-fetching the same mentions. Persist a monotonic cursor (newest processed tweet ID/timestamp) instead of advancing per item.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@listener.py` around lines 156 - 159, Update the mention-processing flow
around the processed_mentions check so the persisted start_time/cursor advances
monotonically to the newest processed mention, rather than being overwritten for
each newest-first item. Track the maximum tweet ID or timestamp across handled
mentions and persist that value only after processing the batch, while
preserving the skip behavior for already processed mentions.
| grok_reply = get_grok_reply(prompt) | ||
| except Exception as exc: | ||
| print(f"Error getting Grok reply for mention {mention.id}: {exc}", flush=True) | ||
| grok_reply = "Processing your tag... (error generating full response)" | ||
| grok_reply = "Sorry, I'm having trouble processing that. Try again or DM me." |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Route all failed Grok results through the apology fallback.
get_grok_reply() returns "Missing XAI_API_KEY." and "Thinking..." for failure-like paths instead of raising. This handler therefore publishes and records those placeholders as successful replies. Return a typed success/error result or raise for these cases before calling create_tweet().
🧰 Tools
🪛 Ruff (0.15.21)
[warning] 172-172: Do not catch blind exception: Exception
(BLE001)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@listener.py` around lines 171 - 174, Update the get_grok_reply flow in the
mention handler so failure-like return values such as “Missing XAI_API_KEY.” and
“Thinking...” are detected before create_tweet(). Route them through the
existing apology fallback and ensure they are not published or recorded as
successful replies; preserve the current exception handling behavior for raised
errors.
| try: | ||
| client.create_tweet( | ||
| text=grok_reply[:280], | ||
| in_reply_to_tweet_id=mention.id, | ||
| ) | ||
| processed_mentions.add(mention_id_str) | ||
| save_processed_mention(mention_id_str) |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Do not treat state-persistence failure as tweet failure.
If create_tweet() succeeds but save_processed_mention() fails, this block logs “Error replying” after already sending the reply. The ID is also added to memory before the disk write, so only the current process suppresses a duplicate; after restart, the reply is sent again. Separate the external write from the durable state commit and retry/fail closed on persistence errors instead of blindly replaying a non-idempotent tweet.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@listener.py` around lines 176 - 182, The mention handler around create_tweet
and save_processed_mention must distinguish successful tweet delivery from
durable state persistence. Do not let save_processed_mention failures enter the
“Error replying” path or treat the tweet as unsent; make persistence failures
retry or fail closed, and only update processed_mentions consistently with a
successful durable commit to prevent replay after restart.
|
Do not resolve the conflicts on this PR. Re-cut the branch from current I audited all five open PRs (#106–#110) against the GitHub compare API. Every one of them is a long-stale branch, not a genuine content conflict:
All five report Why the diffs look absurd#109 is titled "fix: replace invalid CODEOWNERS entries with @groupthinking" but reports +30,880 / −216 across 38 files, with essentially every file showing That is not what the PR intends to change. The branch was cut on 2026-01-27 and opened as a PR on 2026-07-24 — a six-month gap during which The conflicts are therefore real, but they are drift, not disagreement. Resolving them by hand means manually reconciling six months of divergence across 38 files to land what should be a handful of line edits — and every manual resolution is an opportunity to silently revert work that landed on ResolutionFor each PR, the actual intent is small and is captured in only 1–12 commits. Re-apply that intent on top of current git fetch origin
git checkout -b <name>-rebased origin/main
git cherry-pick <the 1-12 real commits> # or simply re-make the edit by handThen open a replacement PR and close the stale one. For #109 specifically, the intended change is a CODEOWNERS edit — that is a few lines, and re-making it by hand against current Recommended disposition:
Root cause to fix going forward: these five PRs were all opened within a 112-second window (22:41:30 → 22:43:22 on 2026-07-24) from branches that were months old. Whatever automation opened them did not rebase first. Adding a staleness check — refuse to open a PR whose branch is more than N commits behind its base — would prevent this class of PR entirely. |
No description provided.