fix(drawio): encode literal tab/newline/CR in content= as numeric char-refs (#507 review)

Review follow-up to the base64→entity-XML content= switch. The whole mxfile XML now lives in one content="..." attribute; XML attribute-value normalization collapses a LITERAL tab/newline/CR to a single space on DOM read (jsdom and the real draw.io editor alike), silently flattening multi-line labels and tab-bearing values that the old base64 form stored verbatim. Finding 1 (data-loss): both encode paths — buildDrawioSvg's xmlEscape (mcp) and the import service's escape — now append 	/
/ after the four &<>" replaces (numeric char-refs survive normalization, as draw.io's own export does). The extractContentAttr regex fallback now decodes those char-refs (hex case-insensitive plus decimal 	/
/) so it agrees with the DOM path; & stays decoded last so an escaped &#x9; reads back as literal text. Finding 2 (dedup): the server's private xmlEscapeAttr is replaced by the shared htmlEscape helper (& < > " ' — a strict superset, the extra ' is harmless in a "-delimited value) wrapped in xmlEscapeContent, which adds the three control-char char-refs on top (htmlEscape does not escape them). Finding 3 (docs): narrow the CHANGELOG healing claim — only a diagram still holding its original correct-UTF-8 base64 (not yet opened/autosaved) is recoverable; one already opened in the editor persisted mojibake at rest and its text is lost. Tests: new mcp round-trip test with literal tab/newline/CR in a value (DOM path, byte-stable) plus a fallback-branch test forcing a malformed wrapper so both decode paths are proven to agree; new server spec asserting char-ref encoding. Mutation-checked: dropping the encode replaces reddens both new mcp tests; dropping only the fallback decode reddens just the fallback test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 03:44:50 +03:00
parent 355bb81a20
commit 1edc8b8dad
5 changed files with 123 additions and 14 deletions
@@ -347,10 +347,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
  as XML-entity-escaped mxfile XML — draw.io's own native form, decoded by the
  DOM as UTF-8 — so labels open intact. The decoder reads both the new
  entity-encoded form and the old base64 form, so existing diagrams still open.
-  *Healing pre-fix diagrams:* any agent-created cyrillic diagram written before
-  this fix is repaired in place by `drawioGet` → `drawioUpdate` with the same
-  XML (rewrites the attachment in the new form); no migration script is needed.
-  (#507)
+  *Healing pre-fix diagrams:* only a diagram that still holds its original
+  (correct-UTF-8) base64 — i.e. one not yet opened/autosaved in the draw.io
+  editor — can be repaired in place by `drawioGet` → `drawioUpdate` with the
+  same XML (rewrites the attachment in the new form); no migration script is
+  needed. A diagram that was already opened in the editor persisted the
+  mojibake at rest, so `drawioGet` reads the already-corrupted text and
+  `drawioUpdate` faithfully rewrites it — that text is lost and is not
+  recoverable by a rewrite. (#507)
 - **A chat with one malformed message part no longer 500s on every turn, and a
  failed send no longer duplicates the user's message.** Incoming client parts
  are now whitelisted to `text` (a forged tool-result part can no longer reach
@@ -76,6 +76,41 @@ describe('ImportAttachmentService.createDrawioSvg (#507)', () => {
    expect(decoded).toBe(drawio);
  });

+  it('encodes literal tab/newline/CR as numeric char-refs, not literal control chars (#507 F1)', async () => {
+    // A literal tab/newline/CR inside the mxfile XML would be collapsed to a
+    // single space by XML attribute-value normalization when the draw.io editor
+    // reads content=, silently flattening multi-line labels and tab-bearing
+    // values. They must be emitted as numeric char-refs instead.
+    const drawio =
+      '<mxfile><diagram name="p">' +
+      '<mxGraphModel><root><mxCell id="0"/>' +
+      '<mxCell id="2" value="col1\tcol2" style="html=1;\nshadow=0" vertex="1" parent="0"/>' +
+      '<mxCell id="3" value="Line1\nLine2\rLine3" vertex="1" parent="0"/>' +
+      '</root></mxGraphModel></diagram></mxfile>';
+    const p = await writeDrawio('ctrl.drawio', drawio);
+
+    const svg = (await call(p)).toString('utf-8');
+    const content = /content="([^"]*)"/.exec(svg)?.[1];
+    expect(content).toBeDefined();
+    // No literal control chars survive in the attribute value.
+    expect(content).not.toMatch(/[\t\n\r]/);
+    // They round-trip as numeric char-refs.
+    expect(content).toContain('&#x9;');
+    expect(content).toContain('&#xa;');
+    expect(content).toContain('&#xd;');
+    // Decoding (char-refs back to literal, entities back) recovers the file.
+    const decoded = content!
+      .replace(/&lt;/g, '<')
+      .replace(/&gt;/g, '>')
+      .replace(/&quot;/g, '"')
+      .replace(/&#39;/g, "'")
+      .replace(/&#x9;/gi, '\t')
+      .replace(/&#xa;/gi, '\n')
+      .replace(/&#xd;/gi, '\r')
+      .replace(/&amp;/g, '&');
+    expect(decoded).toBe(drawio);
+  });
+
  it('escapes XML metacharacters in the drawio payload', async () => {
    const drawio = '<mxfile><diagram name="a &amp; b">"q" &lt;x&gt;</diagram></mxfile>';
    const p = await writeDrawio('meta.drawio', drawio);
@@ -8,6 +8,7 @@ import { createReadStream } from 'node:fs';
 import { promises as fs } from 'fs';
 import { Readable } from 'stream';
 import { getMimeType, sanitizeFileName } from '../../../common/helpers';
+import { htmlEscape } from '../../../common/helpers/html-escaper';
 import { v7 } from 'uuid';
 import { FileTask } from '@docmost/db/types/entity.types';
 import { getAttachmentFolderPath } from '../../../core/attachment/attachment.utils';
@@ -854,7 +855,7 @@ export class ImportAttachmentService {
      // (no UTF-8 step), turning every non-ASCII char (Cyrillic, ё, —) into
      // mojibake; the entity-encoded form is decoded by the DOM as UTF-8 and
      // opens intact. Docmost's own decoder reads both forms.
-      const drawioEscaped = this.xmlEscapeAttr(drawioContent);
+      const drawioEscaped = this.xmlEscapeContent(drawioContent);

      let imageElement = '';
      // If we have a PNG, include it in the SVG
@@ -891,14 +892,20 @@ export class ImportAttachmentService {

  /**
   * Escape a string so it is safe as the value of a double-quoted XML attribute
-   * (the `content=` payload of a `.drawio.svg`). Order matters: `&` first.
+   * (the `content=` payload of a `.drawio.svg`). The shared `htmlEscape` covers
+   * `& < > " '` (a strict superset of what this attribute needs; the extra `'`
+   * escape is harmless in a `"`-delimited value). On top of that, the numeric
+   * char-refs for tab/newline/CR are required: a literal tab/newline/CR inside
+   * an attribute value is collapsed to a single space by XML attribute-value
+   * normalization on DOM read (both our decoder and the real draw.io editor),
+   * silently flattening multi-line labels and tab-bearing values. Char-refs
+   * survive that normalization (#507).
   */
-  private xmlEscapeAttr(s: string): string {
-    return s
-      .replace(/&/g, '&amp;')
-      .replace(/</g, '&lt;')
-      .replace(/>/g, '&gt;')
-      .replace(/"/g, '&quot;');
+  private xmlEscapeContent(s: string): string {
+    return htmlEscape(s)
+      .replace(/\t/g, '&#x9;')
+      .replace(/\n/g, '&#xa;')
+      .replace(/\r/g, '&#xd;');
  }

  private async uploadWithRetry(opts: {
@@ -196,12 +196,23 @@ export function extractContentAttr(svg: string): string {
  // value itself never contains a double-quote (base64 / entity-encoded XML).
  const m = /content="([^"]*)"/.exec(svg);
  if (m) {
-    // Decode the handful of XML entities a raw regex would leave encoded.
+    // Decode the handful of XML entities a raw regex would leave encoded. The
+    // numeric char-refs for tab/newline/CR MUST be decoded here too: the DOM
+    // path above turns them back into the literal control chars, so this
+    // regex fallback has to agree or the two decode paths diverge (#507).
+    // `&amp;` is decoded last so an escaped `&amp;#x9;` reads back as the
+    // literal text `&#x9;`, not a tab.
    return m[1]
      .replace(/&lt;/g, "<")
      .replace(/&gt;/g, ">")
      .replace(/&quot;/g, '"')
      .replace(/&#39;/g, "'")
+      .replace(/&#x9;/gi, "\t")
+      .replace(/&#9;/g, "\t")
+      .replace(/&#xa;/gi, "\n")
+      .replace(/&#10;/g, "\n")
+      .replace(/&#xd;/gi, "\r")
+      .replace(/&#13;/g, "\r")
      .replace(/&amp;/g, "&");
  }
  throw new Error("drawio: SVG has no content= attribute to decode");
@@ -342,7 +353,15 @@ function xmlEscape(s: string): string {
    .replace(/&/g, "&amp;")
    .replace(/</g, "&lt;")
    .replace(/>/g, "&gt;")
-    .replace(/"/g, "&quot;");
+    .replace(/"/g, "&quot;")
+    // A literal tab/newline/CR inside an attribute value is collapsed to a
+    // single space by XML attribute-value normalization on DOM read (both jsdom
+    // here and the real draw.io editor), silently flattening multi-line labels
+    // and tab-bearing values. Numeric char-refs survive that normalization, so
+    // emit them the way draw.io's own native export does (#507).
+    .replace(/\t/g, "&#x9;")
+    .replace(/\n/g, "&#xa;")
+    .replace(/\r/g, "&#xd;");
 }

 // --- normalization + hash --------------------------------------------------
@@ -435,3 +435,47 @@ test("#507 negative: non-ASCII in a cell value AND in the title round-trip clean
  assert.ok(nameMatch, "diagram name attribute present");
  assert.equal(unescapeAttr(nameMatch[1]), title);
 });
+
+// --- #507 F1: literal tab/newline/CR in an attribute value survive the
+// content= round-trip. The whole mxfile XML lives in one content="..." attr;
+// XML attribute-value normalization collapses a LITERAL tab/newline/CR to a
+// single space on DOM read, so the escape must emit numeric char-refs instead.
+const CTRL_MODEL =
+  "<mxGraphModel><root>" +
+  '<mxCell id="0"/><mxCell id="1" parent="0"/>' +
+  '<mxCell id="2" value="col1\tcol2" style="html=1;\nshadow=0" vertex="1" parent="1">' +
+  '<mxGeometry x="10" y="10" width="120" height="60" as="geometry"/></mxCell>' +
+  '<mxCell id="3" value="Line1\nLine2\rLine3" vertex="1" parent="1">' +
+  '<mxGeometry x="10" y="100" width="120" height="60" as="geometry"/></mxCell>' +
+  "</root></mxGraphModel>";
+
+test("#507 F1: literal tab/newline/CR in a value round-trip byte-stable (DOM decode)", () => {
+  const svg = buildDrawioSvg(CTRL_MODEL, "<g/>", { width: 200, height: 200 });
+  const content = /content="([^"]*)"/.exec(svg)[1];
+  // The tab/newline/CR must be emitted as numeric char-refs, never as literal
+  // control chars (which DOM attribute-value normalization would eat).
+  assert.ok(
+    !/[\t\n\r]/.test(content),
+    "no literal tab/newline/CR survive in the content= attribute",
+  );
+  assert.ok(
+    content.includes("&#x9;") &&
+      content.includes("&#xa;") &&
+      content.includes("&#xd;"),
+    "tab/newline/CR are emitted as numeric char-refs",
+  );
+  // Full DOM decode recovers the model byte-for-byte, control chars intact.
+  assert.equal(decodeDrawioSvg(svg), CTRL_MODEL);
+});
+
+test("#507 F1: regex fallback decodes tab/newline/CR char-refs (agrees with DOM path)", () => {
+  const svg = buildDrawioSvg(CTRL_MODEL, "<g/>", { width: 200, height: 200 });
+  const content = /content="([^"]*)"/.exec(svg)[1];
+  // A bare `&` makes the SVG wrapper malformed, forcing extractContentAttr onto
+  // its regex fallback branch. That branch must decode the tab/newline/CR
+  // char-refs exactly like the DOM path, or the two decoders diverge.
+  const malformedSvg = `<svg content="${content}">&</svg>`;
+  assert.equal(decodeDrawioSvg(malformedSvg), CTRL_MODEL);
+  // The well-formed (DOM) path yields the identical result.
+  assert.equal(decodeDrawioSvg(svg), CTRL_MODEL);
+});