Opportunity
After every run, Skipper knows exactly which tests it suppressed, why, and until when. Today that information goes nowhere. The opportunity: surface a quarantine debt score in every CI run — making the cost of hiding tests visible and creating accountability pressure that no other tool in this space provides.
Proposed feature
Emit a structured quarantine report at the end of every test session, targeting two outputs:
- GitHub Actions job summary (via
GITHUB_STEP_SUMMARY)
- A local
skipper-report.json artifact for archiving and downstream tooling
The report includes: number of currently suppressed tests, tests expiring this week, tests re-enabled in this run, and a total quarantine-days of debt score (sum of disabled_until - today across all active rows).
Wire the emission into:
skipper-pytest — via pytest_sessionfinish
skipper-unittest — via a tearDownModule / runner hook on SkipperTestCase
skipper-playwright — via the existing session-end hook in SkipperSyncTest
import json, os
from pathlib import Path
def emit_summary(report: dict) -> None:
md = build_markdown_summary(report)
summary_file = os.environ.get("GITHUB_STEP_SUMMARY")
if summary_file:
Path(summary_file).open("a").write(md)
else:
print(md)
Path("skipper-report.json").write_text(json.dumps(report, indent=2, default=str))
The helper lives in skipper-core and is invoked once per session by each integration.
Why this is the unique moat
The quarantine-days of debt number grows when tests are disabled and shrinks when they expire. A team with 200 quarantine-days of debt is hiding a lot of reality. That number in every PR comment is a forcing function that costs zero additional tooling — and no other tool in this space does it.
Acceptance criteria
- After every session, a markdown summary is written to
GITHUB_STEP_SUMMARY if the env var is set
- If
GITHUB_STEP_SUMMARY is not set, the summary is printed to stdout
skipper-report.json is written with the full suppression state of the run
- Report includes: currently suppressed count, expiring this week, re-enabled this run, quarantine-days of debt
- Quarantine-days of debt = sum of
(disabled_until - today).days across all active disabled rows
- Implemented once in
skipper-core, invoked by skipper-pytest, skipper-unittest, skipper-playwright
Effort estimate
~80 lines in skipper-core plus ~5 lines per integration package, stdlib only.
Opportunity
After every run, Skipper knows exactly which tests it suppressed, why, and until when. Today that information goes nowhere. The opportunity: surface a quarantine debt score in every CI run — making the cost of hiding tests visible and creating accountability pressure that no other tool in this space provides.
Proposed feature
Emit a structured quarantine report at the end of every test session, targeting two outputs:
GITHUB_STEP_SUMMARY)skipper-report.jsonartifact for archiving and downstream toolingThe report includes: number of currently suppressed tests, tests expiring this week, tests re-enabled in this run, and a total quarantine-days of debt score (sum of
disabled_until - todayacross all active rows).Wire the emission into:
skipper-pytest— viapytest_sessionfinishskipper-unittest— via atearDownModule/ runner hook onSkipperTestCaseskipper-playwright— via the existing session-end hook inSkipperSyncTestThe helper lives in
skipper-coreand is invoked once per session by each integration.Why this is the unique moat
The quarantine-days of debt number grows when tests are disabled and shrinks when they expire. A team with 200 quarantine-days of debt is hiding a lot of reality. That number in every PR comment is a forcing function that costs zero additional tooling — and no other tool in this space does it.
Acceptance criteria
GITHUB_STEP_SUMMARYif the env var is setGITHUB_STEP_SUMMARYis not set, the summary is printed to stdoutskipper-report.jsonis written with the full suppression state of the run(disabled_until - today).daysacross all active disabled rowsskipper-core, invoked byskipper-pytest,skipper-unittest,skipper-playwrightEffort estimate
~80 lines in
skipper-coreplus ~5 lines per integration package, stdlib only.