feat(ci): generates a GitHub QA Dashboard crossing Penpot issues with automated tests

[daniel-herrero]

Taiga URL

Taiga Ticket: 1623

Description

This PR resolves the lack of visibility into how well our automated regression suite (Qase) actually covers the areas of Penpot where real bugs keep showing up (GitHub Issues).

What problem are you trying to solve?

Today, deciding "where should we invest in more automated tests?" is a judgment call — nobody can easily cross-reference the Qase test inventory against the volume of bugs/features reported per area of the app. This PR adds a standalone pipeline (scripts/github-dashboard/) plus a scheduled GitHub Actions workflow that generates a self-contained HTML dashboard crossing:

• Qase API (/v1/suite/PENPOT and /v1/case/PENPOT): test suite hierarchy, full test inventory, and regression/automation status.
• GitHub Project v2 API (penpot org, via GraphQL — the only way to get the Status field): bugs/features/enhancements, labels, dates.

... so we can see, per area, how many tests exist vs. how many bugs land there, and make a data-backed call on where to prioritize automation.

Solution

How did you solve the problem?

• github_dashboard.ts — dependency-free TypeScript script (Node ≥18, native fetch) that pulls both data sources, maps each GitHub issue to a Penpot "area" via mapping.json (exact label match → keyword regex → fallback), deduplicates entities (Qase pagination/export duplicates are a known issue, documented in the README), and writes decision-matrix.csv, mapped-bugs.csv, mapped-features.csv, monthly-trend.csv, regression-tests.csv, and a self-contained YYYY-MM-DD-github-dashboard.html into out/.
• mapping.json — all issue→area mapping logic lives here so it can be maintained without touching code (label map, keyword regexes, exclude labels).
• dashboard_template.html — the HTML/CSS/JS shell the script injects data into (no external dependencies, so it works as a plain file or from S3). It also includes links to the dashboards from previous weeks (past runs menu on the top right corner), and links to the test cases in Qase and GitHub issues (by clicking on the last table numbers).
• .github/workflows/github-dashboard.yml — runs every Monday 04:30 UTC (cron is always evaluated on the default branch) or on-demand via workflow_dispatch (any branch, with an s3_prefix input to avoid clobbering the real report while testing). Installs deps, runs the script, zips the whole directory (inputs + out/) for a "download raw data" link, publishes both a dated snapshot and a latest/ pointer to S3, maintains a history.json so the dashboard's own "past runs" dropdown can list previous weeks, and posts a Mattermost notification with the result.
• Follow-up commits fix regression/automated test counts and add links from the dashboard back to Qase/GitHub, fix area mapping for horizontal WASM-related issues, and modularize the build step for readability.
• Adding the QA_GH_PROJECT_TOKEN environment secret on PRE (it requires a project:read permisison), and reusing the already existing QASE_API_TOKEN.

How to test

• Check the code
• Update the test case in Qase (Automation Status field or steps changed)
• It complies with the test conventions
• There are no missing snapshots
• The tests run OK
• Confirm the pull_request trigger commit is dropped before merging to main
• Optionally run the workflow manually (Actions tab → "GitHub dashboard" → Run workflow) against this branch with s3_prefix: github-dashboard-test to sanity-check the generated dashboard without touching the real latest/ output

Screenshots 📸 (optional)

The dashboard, the raw data CSVs in S3, and the MM message

Anything Else?

This is a standalone internal tooling script (not part of the Playwright test suite itself) — it lives under scripts/github-dashboard/ and requires QASE_API_TOKEN/QASE_TOKEN and a classic GITHUB_TOKEN/QA_GH_PROJECT_TOKEN PAT with read:project scope (the default Actions token can't read org Projects v2). See scripts/github-dashboard/README.md for full setup, token scopes, and the "Possible Enhancements" / "Data Quality Notes" sections.