Skip to content

Keep the line breaks in a stored FOS report, and escape it on the way out - #1223

Merged
mastacontrola merged 1 commit into
working-1.6from
keep-fos-report-line-breaks
Aug 19, 2026
Merged

Keep the line breaks in a stored FOS report, and escape it on the way out#1223
mastacontrola merged 1 commit into
working-1.6from
keep-fos-report-line-breaks

Conversation

@mastacontrola

Copy link
Copy Markdown
Member

Follow-up to #1217. Widening MAX_TEXT to 8192 bytes bought less than it looked like: _sanitize() stripped every control character including the newline, so 8K of trace arrived as one unbroken line — barely more readable than the 500 characters it replaced.

Where the flattening belongs

It was there for a real reason, just not for the row. An embedded newline lets a caller forge a second entry — true of a chat notification, and of fosreports.log, whose entries are one timestamped line each and which gets read with tail. Not true of a database row rendered in a modal.

destination line breaks
taskLog.logText kept
Slack / ntfy / pushbullet Reason flattened, then cut to MAX_REASON
fosreports.log flattened

_sanitize() now keeps \n and strips everything else. Two details that are easy to get wrong and are pinned by tests:

  • \r\n and lone \r are normalised to \n first. Without that the CR is merely replaced by a space, so every line of a DOS-ended report — which is most of them — stores with trailing whitespace.
  • The invalid-UTF-8 fallback is an explicit byte range, not [[:cntrl:]]. That class includes the newline, so a machine with the wrong locale would silently lose the shape of its report while every other machine kept it.

_flatten() is flattening applied before the MAX_REASON cut, so the 500 characters are spent on text rather than on indentation.

Two display fixes fall out of it

The logs grid's Message column had no render at all. DataTables writes cell content with innerHTML, so a column without one is an HTML sink — and this is the single column fed by taskerror.class.php, an endpoint FOS reaches without authenticating. Every other column in that grid already escaped; this one was missed. It now escapes, and flattens for the one-line preview.

The modal was <pre class="mb-0 text-wrap">. Bootstrap defines .text-wrap as white-space: normal !important, which overrides the <pre> and collapses exactly the line breaks now being stored. Replaced with pre-wrap, which keeps them and still wraps long lines.

FOG_BCACHE_VER 287 → 288 for the JS change.

Verification

Against a copy of the live database — a CRLF report carrying terminal escapes and a NUL:

stored:    3 line breaks, no CR, no NUL, no ESC byte
flattened: 0 line breaks

Eight mutations across both gates, all killed. Three survived the first pass, all the same way — an assertion looking at more than its subject. The escaping check searched the whole file for targets: 5, which occurs in four grids here, and had been reporting on a different pane's column entirely; it is now scoped to buildLogs() and bounded by the neighbouring columnDef. The other two simply had no test case carrying a newline through the path they guarded.

Full suite: 71 passed, 0 failed.

On dev-branch

The PHP half is ported in the companion PR. The grid and the modal are not: 1.5 has no logs tab, no js/fog/task/, and no taskLog viewer at all, so there is nothing there to escape or to wrap. The row is stored faithfully either way.

🤖 Generated with Claude Code

https://claude.ai/code/session_013mJVe4CpK3rRbi9H5GubXd

… out

Widening MAX_TEXT to 8192 bytes bought less than it looked like: _sanitize()
stripped every control character including the newline, so 8K of trace
arrived as one unbroken line -- barely more readable than the 500 characters
it replaced.

The flattening was there for a real reason, but not for the row. An embedded
newline lets a caller forge a second entry, and that is true of a chat
notification and of fosreports.log, whose entries are one timestamped line
each and which `tail` is read as. It is not true of a database row rendered
in a modal. So the guard moved to the two destinations that need it:

- _sanitize() keeps \n and strips everything else, after normalising \r\n
  and lone \r to \n. Without that normalisation the CR is merely replaced
  by a space, so every line of a DOS-ended report stores with trailing
  whitespace. The invalid-UTF-8 fallback had to be spelled out as a byte
  range rather than [[:cntrl:]], because that class includes the newline --
  otherwise a machine with the wrong locale silently loses the shape of its
  report while every other machine keeps it.

- _flatten() collapses whitespace to single spaces, and the notification and
  the log line each call it. The notification is flattened BEFORE its
  MAX_REASON cut, so the 500 characters are spent on text rather than on
  indentation.

Two display fixes fall out of it:

- The logs grid's Message column had no render at all. DataTables writes
  cell content with innerHTML, so a column without one is an HTML sink --
  and this is the single column fed by taskerror.class.php, an endpoint FOS
  reaches without authenticating. Every other column in that grid already
  escaped; this one was missed. It now escapes, and flattens for the
  one-line preview.

- The modal rendered the message in `<pre class="mb-0 text-wrap">`, and
  Bootstrap defines .text-wrap as `white-space: normal !important`, which
  overrides the <pre> and collapses exactly the line breaks now being
  stored. Replaced with pre-wrap, which keeps them and still wraps long
  lines.

FOG_BCACHE_VER 287 -> 288 for the JS change.

Verified against a copy of the live database: a CRLF report with escapes and
a NUL stores with its 3 line breaks and no CR, NUL or ESC byte, and the same
text flattens to 0 line breaks for the notification and the log.

Eight mutations across both gates, all killed. Three survived the first
pass, all the same way -- an assertion looking at more than its subject.
The escaping check searched the whole file for `targets: 5`, which occurs in
four grids here, and was reporting on a different pane's column entirely; it
is now scoped to buildLogs() and bounded by the neighbouring columnDef. The
other two had no test case carrying a newline through the path they guarded.

Co-Authored-By: Claude <noreply@anthropic.com>
@mastacontrola
mastacontrola merged commit 94d0bc1 into working-1.6 Aug 19, 2026
3 checks passed
@mastacontrola
mastacontrola deleted the keep-fos-report-line-breaks branch August 19, 2026 17:54
mastacontrola added a commit that referenced this pull request Aug 19, 2026
v1.6.11 carries FOGProject/fog-plugins#21, which was merged the morning after
v1.6.10 was cut and so has never reached a server.

That change matters more than its own PR made it sound, because the FOS
reporting work landed in between. #1206/#1211/#1217/#1223 give a failed task a
stored, multi-line report of what FOS actually said, and TaskError sends the
flattened opening of it as HOST_IMAGE_FAIL's Reason -- but every bundled
listener on v1.6.10 ignores that key and pushes the fixed string "This host
has failed to image".

Confirmed live before cutting the release. A report whose stored row read

    fog.download: failed to restore partition 2
      partclone.ntfs: /dev/sda2 is busy
      ERROR: win11-split part 2 checksum mismatch
      exit code 1 (fog.download)

pushed, in its entirety: "fos-deploy-test Failed" / "This host has failed to
image". So the whole point of storing the trace stopped at the server.

One line, because the sha256 is fetched from the release alongside the
tarball rather than pinned here; bin/fetch-plugins.sh reads this constant and
verifies what it downloads against it. Verified end to end on this pin:
"Fetching plugins v1.6.11" -> "Plugins at v1.6.11", and all three imagefail
listeners in the fetched tree read Reason.

The 1.5 line keeps its plugins in-tree and has no pin, so its equivalent is a
code change: #1226.

Co-authored-by: JJ Fullmer <7743340+darksidemilk@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants