feat(ci): triage v2 task-level tracking and auto-close of resolved failures

[lugarbos]

Done Definition Checks

Description

The first iteration of the triage automation worked, but real use surfaced four gaps. Every line in the digest linked to the release user story instead of the specific task, so "Taiga #1683" appeared repeated on every cluster and reviewers still had to hunt for the right task. Resolved failures were only flagged, leaving manual closing work. Deleting the release story in Taiga (a natural way to try to "reset") left the state file behind, producing a confusing situation where re-runs recreated an empty story with no tasks. And a run failed against api.taiga.io with an HTML 403 — an intermittent Cloudflare edge block of GitHub runner traffic, unrelated to our credentials.

Solution

• Task-level tracking. When a cluster's task is created, its Taiga task id, ref, and subject are now stored in the per-release state. Every digest and Mattermost line — new, known, and resolved clusters — links to the cluster's own task (.../task/) instead of the story; the story link remains only as the single entry point in the Mattermost verdict line. Resolved lines also show the task's subject so "what went green" is readable without clicking.

• Auto-close of resolved tasks. A cluster green for 3 consecutive triaged runs is now closed automatically: the script resolves the project's "Closed" task status and the qa.integrations.bot user via the API, then updates the task with that status, that assignee, and an explanatory comment. Closed fingerprints are removed from state, so a failure that returns later becomes a fresh cluster with a fresh task. Both names are configurable (TAIGA_CLOSE_ASSIGNEE, TAIGA_CLOSED_STATUS). Degradation is graceful at every step: unknown status name → skip with a log line; bot not a project member → close without reassigning; a single failed close → logged and retried on the next run. Mattermost distinguishes Auto-closed from Resolved — close manually (the latter covers file-mode clusters, which share tasks per spec file and are never auto-closed, and pre-upgrade state entries that never recorded a task ref).

• Story deletion is now a supported rebuild. If the stored release story no longer exists in Taiga, the script rebuilds it and recreates tasks for known clusters (their tasks died with the story), while preserving each cluster's failure history ("failing N runs in a row" counters survive). The file-mode task map is reset accordingly.

• Self-service reset. New reset_state boolean input on the workflow: when ticked, the run deletes the tag's state file from S3 before triaging, so the whole release starts from scratch — no AWS access needed by the person triggering. Scoped strictly to the given tag; deleting the old story in Taiga remains a deliberate human action.

• Taiga edge hardening. All Taiga requests now send a User-Agent (Cloudflare commonly 403-blocks UA-less requests with an HTML page). An HTML-bodied 403 is detected, explained in the error message ("blocked before reaching Taiga / check TAIGA_URL"), and retried twice with backoff — while a genuine JSON 403 (real permission problem) still fails fast.

• Rename. The workflow is now Report triage (.github/workflows/report-triage.yml, previously release-triage.yml). Internal concepts (release tag, release- Taiga tag, state filename) are unchanged, so existing state remains valid.

• Migration note for in-flight tags

• Tasks created before this change never recorded their task refs, so their clusters keep linking to the story and cannot be auto-closed. For any tag still in testing (e.g. 2.18): delete its story in Taiga and trigger with reset_state ticked — the rebuild records task refs for everything and the new behavior applies fully. Tags started after this PR need nothing.

How to test

1. Trigger Report triage with a test tag and reset_state ticked → verify one story, one task per cluster, and that digest/Mattermost lines link to individual tasks.
2. Re-trigger without reset → known clusters keep their task links; no duplicates.
3. Simulate a resolution (or wait for one): after 3 green triaged runs the task should appear as Closed and assigned to qa.integrations.bot, and Mattermost should list it under "Auto-closed".
4. Delete the story (not the state) and re-trigger → story and all tasks rebuilt, history counters preserved (Rebuilding tasks for N known cluster(s) in the log).

Prerequisite check: the Taiga project must have a task status named "Closed" (or set TAIGA_CLOSED_STATUS), and qa.integrations.bot must be a project member.

[daniel-herrero]

LGTM! 👍