Skip to content

ansiescape: cover all control-sequence introducers, not just CSI/OSC#1978

Closed
WatchTree-19 wants to merge 1 commit into
NVIDIA:mainfrom
WatchTree-19:ansiescape-introducer-coverage
Closed

ansiescape: cover all control-sequence introducers, not just CSI/OSC#1978
WatchTree-19 wants to merge 1 commit into
NVIDIA:mainfrom
WatchTree-19:ansiescape-introducer-coverage

Conversation

@WatchTree-19

Copy link
Copy Markdown
Contributor

Fixes #1977.

The gap

Both ansiescape detectors built their needles from a hardcoded suffixes = {"[", "]"}, so only CSI and OSC were recognised. ESC c (RIS) and the DCS/SOS/PM/APC string controls — plus their C1 single-byte forms 0x90/0x98/0x9e/0x9f — scored 0.0 in both Raw and Escaped. APC is not theoretical: it is live in the wild via the kitty graphics protocol.

As #1977 notes, this is a coverage boundary rather than a live false negative — all seven existing LIVE_PAYLOADS still score 1.0 today. Before/after on the uncovered set:

introducer before after
RIS, DCS, SOS, PM, APC 0.0 1.0
C1 DCS/SOS/PM/APC 0.0 1.0

Why it isn't just a wider suffix list

Widening the list naively introduces false positives, which is the substantive part of this change. StringDetector matches case-insensitively (case_sensitive defaults to False), so crossing the alphabetic introducers with the shell-style \e spelling produces the needles \ec, \ep and \ex — which match benign text like \echo, \epsilon and \export. I hit this by way of my own test rather than by inspection.

So the introducers are split:

  • INTRODUCERS ([, ], ^, _) — punctuation, safe to cross with every command spelling.
  • ALPHA_INTRODUCERS (c, P, X) — pinned for the Escaped detector against the unambiguous command spellings only, derived from ESCAPED_COMMANDS so they track future additions automatically. RAW_COMMANDS are control bytes rather than text and carry no such risk, so Raw takes the full set.

RIS

RIS is covered for detection and included in ESCAPED_PAYLOADS as literal text, but deliberately kept out of LIVE_PAYLOAD_TOKENS. Unlike every other entry there — all inert, incomplete introducers — ESC c is a complete two-byte terminal reset, and the module note is explicit that these payloads land in both garak and remote logs. Happy to add it if you'd rather have the coverage.

ST (0x9c), charset designation and DECALN remain out of scope per the issue.

Verification

  • tests/detectors/test_detectors_ansiescape.py and tests/probes/test_probes_ansiescape.py — 11 passed.
  • New tests pin each newly covered introducer in both raw and escaped form, assert the two detectors do not cross-fire, and hold benign text (including the \e-prefixed words above) at 0.0.
  • Existing raw/escaped assertions are unchanged; I re-verified the LIVE_PAYLOADSRaw-only / ESCAPED_PAYLOADSEscaped-only separation still holds with 0 violations, including when simulated against the widened ESCAPED_COMMANDS from ansiescape: detect alternative escaped ESC encodings (2-digit octal, curly/8-digit unicode) #1929.
  • black clean.

Note on overlap with #1929

This touches ansi.py and test_detectors_ansiescape.py, as does my open #1929. The two are orthogonal axes — #1929 widens the ESC encoding set (ESCAPED_COMMANDS), this widens the introducer set — and they compose, since needles are the cross-product. I checked this combination explicitly for false positives (clean, above). Whichever lands first, I'll rebase the other; the textual conflicts are appends to ESCAPED_PAYLOADS and to the end of the test file.

The ansiescape detectors built their needles from a hardcoded
suffixes = {[, ]}, so only CSI and OSC were recognised. ESC c (RIS)
and the DCS/SOS/PM/APC string controls -- plus their C1 single-byte forms
0x90/0x98/0x9e/0x9f -- scored 0.0 in both Raw and Escaped, leaving the
module narrower than the attack class its docstring describes. APC in
particular is live in the wild via the kitty graphics protocol.

Widening the suffix list naively introduces false positives, so the
introducers are split in two:

- INTRODUCERS ([, ], ^, _) are punctuation and safe to cross with every
  command spelling.
- ALPHA_INTRODUCERS (c, P, X) are not. StringDetector matches
  case-insensitively (case_sensitive defaults to False), so crossing them
  with the shell-style \e spelling yields the needles \ec, \ep and \ex,
  which match benign text such as \echo, \epsilon and \export. For the
  Escaped detector these are instead pinned against the unambiguous
  command spellings, derived from ESCAPED_COMMANDS so they track any
  future additions. RAW_COMMANDS are control bytes rather than text and
  carry no such risk, so Raw takes the full set.

RIS is covered for detection and appears in ESCAPED_PAYLOADS as literal
text, but is deliberately kept out of LIVE_PAYLOAD_TOKENS: unlike every
other entry there it is a complete two-byte terminal reset rather than an
inert introducer, and per the module note these payloads land in both
garak and remote logs.

ST (0x9c), charset designation and DECALN remain out of scope.

Tests pin each newly covered introducer in both raw and escaped form,
assert the two detectors do not cross-fire, and keep benign text --
including the \e-prefixed words above -- at 0.0. The pre-existing
LIVE/ESCAPED payload separation is unchanged.

Signed-off-by: WatchTree-19 <119982314+WatchTree-19@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ansiescape: detector and payloads cover only CSI and OSC introducers

1 participant