Add synthetic DOCX fidelity corpus and stored export projections
16 files changed,
+382
−0
docs/superpowers/specs/2026-09-08-docx-fidelity-corpus.md+40 −0fixtures/docx/alternate-body.docxbinaryfixtures/docx/core.docxbinaryfixtures/docx/core.expected.json+1 −0fixtures/docx/core.expected.md+10 −0fixtures/docx/core.update.binbinaryfixtures/docx/features.expected.json+1 −0fixtures/docx/generate-update.mjs+16 −0fixtures/docx/generate.py+47 −0fixtures/docx/image.docxbinaryfixtures/docx/link.docxbinaryfixtures/docx/manifest.json+26 −0fixtures/docx/numbered-heading.docxbinaryfixtures/docx/table.docxbinaryinternal/docs/docx_projection_test.go+49 −0internal/export/docx_corpus_test.go+192 −0
CI passed — run details
diff --git a/docs/superpowers/specs/2026-09-08-docx-fidelity-corpus.md b/docs/superpowers/specs/2026-09-08-docx-fidelity-corpus.md| new file mode 100644 |
| index 0000000..1fb271c |
| --- /dev/null |
| +++ b/docs/superpowers/specs/2026-09-08-docx-fidelity-corpus.md |
| @@ -0,0 +1,40 @@ |
| +# DOCX fidelity corpus — internal review proposal |
| + |
| +Scope: synthetic fixtures and regression evidence only, based main87662f5b246004aca604a4b040cb6691cfd7f130. No importer/exporter implementation, new UI/copy, schema change, merge or deployment. Existing import branch worktree-import-documents e64fcc7 and discussion20 remain the owners. Frozen links293ccd0 and Estilo candidates remain untouched. |
| + |
| +## Actual capability boundary |
| + |
| +Main has no internal/importer, POST /documents/import, DOCX parser or DOCX export route. Its real exporters are Markdown and PDF. This absence is recorded, not encoded as a test that merely expects404. The existing import branch implements Rust canonical JSON→Y.XmlFragment task0 only; its planned store/HTTP/parser work is unimplemented and DOCX is plan2. Its recorded fragment_text/ABI-versus-snapshot-version rulings remain pending. This proposal neither changes those rulings nor bypasses Docs' spec-before-plan and whole-branch Fable gates. |
| + |
| +Main's editor inherits basic link/image nodes; its server/export models do not preserve their full attributes. Tables are not in the current editor schema. The actual Markdown/PDF projection probe on main recorded: link label survives but target disappears; table cell text is flattened to paragraphs; image disappears. PDF returns no substitution report for those losses. This was a temporary measurement probe, not a passing regression that would require loss forever. Evidence /tmp/docs-docx-loss-probe.log. Frozen link work owns that first correction. No DOCX input was parsed to obtain these projection results. |
| + |
| +Main spec§11.2 requires named, located import loss reporting; §11.3 promises clean DOCX and Markdown lossless within its subset. Import discussion20's revision3 specifies warning enums and structured locators with no document text/link targets in reports. It currently proposes warned degradation for links/tables/images. Future review must reconcile that subset with the newer link implementation and full-suite migration expectations; do not quietly turn warned degradation into claimed full fidelity or silently flatten a document. |
| + |
| +## Checked-in corpus |
| + |
| +All files in fixtures/docx are generated from synthetic literals; no external documents, network downloads or customer content. ZIP entries have fixed metadata. generate.py creates valid XML parts, content types and OPC relationships, including styles/numbering. generate-update.mjs creates a separate supported-subset Yjs projection via pinned upstream ProseMirror/Yjs packages; it is not a DOCX importer and does not claim to reuse the complete editor schema. |
| + |
| +- core.docx: heading1, paragraph, bold/italic runs, nested bullet list and decimal list. core.expected.json is the independent future-import structural oracle; core.expected.md specifies current Markdown projection; core.update.bin is the separate upstream-authored Yjs projection. |
| +- alternate-body.docx: identical body at word/document2.xml, selected by package officeDocument relationship. A fixed word/document.xml parser must fail this case. |
| +- numbered-heading.docx: Heading1 plus numPr on one paragraph. The expectation remains heading1, not a list item, per import revision3. |
| +- link.docx: Reference label and external relationship to https://example.test/reference. Preserve the target or produce the reviewed located warning/refusal; never fetch the URL during import. |
| +- table.docx: two rows and two columns with distinct cells. Preserve geometry/cell ordering, or explicitly report the reviewed degradation; success with flattened text and no report is failure. |
| +- image.docx: embedded synthetic one-pixel PNG, drawing relationship, extent and alt text. Preserve media/dimensions/alt and per-person custody, or explicitly report the reviewed unsupported construct. No external image fetching. |
| + |
| +features.expected.json captures the non-core semantic expectations without inventing future asset IDs, image upload routes or table node APIs. The unit validator checks ZIP safety/uniqueness, XML well-formedness, body relationship resolution, internal targets, the feature-bearing source shapes and PNG dimensions. This is structural OPC evidence, not certification by Word/LibreOffice or proof of DOCX round-trip pagination. |
| + |
| +## What the real tests certify |
| + |
| +internal/export/docx_corpus_test.go passes the supported expected tree through the actual Markdown/PDF exporters. Exact Markdown checks heading/list nesting and marks; PDF text extraction checks all source words through the generated PDF's text maps. internal/docs/docx_projection_test.go applies the upstream Yjs projection through real Store.ApplyUpdate, then calls authenticated export.md/export.pdf routes and checks owner isolation. The Markdown result matches the structural expectation. The route's PDF media/header check is not presented as a PDF semantic test; the exporter-level extraction test supplies that evidence. |
| + |
| +Mutation evidence: deleting heading markers in actual Markdown output fails both projection tests for changed content, not a route error. Deleting the PDF content fails copied-text preservation, and removing the link relationship mode fails the structural fixture test. No test claims the nonexistent DOCX importer round-tripped anything. Existing import owner's Rust→Yjs cross-implementation gate is a separate dependency and must eventually accept these canonical expectations, with the actual shared editor schema rather than a drifting clone. |
| + |
| +## Implementation proposal to review next |
| + |
| +1. Resolve or explicitly reaffirm discussion20's two recorded rulings, then finish the reviewed import mechanism on its existing branch. Preserve safe JS client IDs, the real Store.ApplyUpdate path, sealed content, report-write failure cleanup and per-person grants. A report must not leak document text or URL targets into plaintext metadata. |
| +2. Connect a bounded Go ZIP/XML DOCX parser to this corpus. Resolve officeDocument and its relationship parts rather than fixed filenames; validate duplicate/escaping entries, decompressed size/part/depth budgets, numbering/styles, heading-over-numPr precedence and explicit XML breaks. Do not follow external relationships. Reject malformed/ambiguous packages without creating a partial document. |
| +3. For each supported input, compare canonical semantic output to checked-in expectations, build through the reviewed Rust importer and read through Yjs/real editor schema, then store/reload/export through actual routes. Content-equivalence expectations must ignore irrelevant ZIP order, namespace prefixes and relationship IDs while retaining node ordering, list depth, marks and destinations. |
| +4. Add tables/images only behind independently settled schema/media-custody contracts. Reuse the frozen link contract once merged; do not create a second URL policy. Before those capabilities exist, enforce the reviewed visible report/refusal path and transactional cleanup. This evidence branch does not choose a new policy for loss acceptance. |
| +5. Implement DOCX export as a separate reviewed slice, then add import→stored Yjs→DOCX→reimport semantic comparisons using this same corpus. Until that exists, report export parity as missing. Formatting beyond these synthetic cases, comments, page setup, complex numbering, headers/footers and tracked changes need additional corpus cases and their own settled scope; no 'replaces Word' claim follows from this small corpus. |
| + |
| +No deployment is required for this evidence deliverable. Product implementation and its generated UI artifacts still require their normal independent review/copy/CI/canary gates. |
diff --git a/fixtures/docx/alternate-body.docx b/fixtures/docx/alternate-body.docx| new file mode 100644 |
| index 0000000..bcd962c |
| Binary files /dev/null and b/fixtures/docx/alternate-body.docx differ |
diff --git a/fixtures/docx/core.docx b/fixtures/docx/core.docx| new file mode 100644 |
| index 0000000..97b7395 |
| Binary files /dev/null and b/fixtures/docx/core.docx differ |
diff --git a/fixtures/docx/core.expected.json b/fixtures/docx/core.expected.json| new file mode 100644 |
| index 0000000..b2d9067 |
| --- /dev/null |
| +++ b/fixtures/docx/core.expected.json |
| @@ -0,0 +1 @@ |
| +{"type":"doc","content":[{"type":"heading","attrs":{"level":1},"content":[{"type":"text","text":"Section"}]},{"type":"paragraph","content":[{"type":"text","text":"Plain "},{"type":"text","text":"bold","marks":[{"type":"strong"}]},{"type":"text","text":" "},{"type":"text","text":"italic","marks":[{"type":"em"}]}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"type":"text","text":"Outer"}]},{"type":"bullet_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"type":"text","text":"Inner"}]}]}]}]}]},{"type":"ordered_list","content":[{"type":"list_item","content":[{"type":"paragraph","content":[{"type":"text","text":"First"}]}]},{"type":"list_item","content":[{"type":"paragraph","content":[{"type":"text","text":"Second"}]}]}]}]} |
diff --git a/fixtures/docx/core.expected.md b/fixtures/docx/core.expected.md| new file mode 100644 |
| index 0000000..603723f |
| --- /dev/null |
| +++ b/fixtures/docx/core.expected.md |
| @@ -0,0 +1,10 @@ |
| +# Section |
| + |
| +Plain **bold** *italic* |
| + |
| +- Outer |
| + |
| + - Inner |
| + |
| +1. First |
| +2. Second |
diff --git a/fixtures/docx/core.update.bin b/fixtures/docx/core.update.bin| new file mode 100644 |
| index 0000000..e982d0e |
| Binary files /dev/null and b/fixtures/docx/core.update.bin differ |
diff --git a/fixtures/docx/features.expected.json b/fixtures/docx/features.expected.json| new file mode 100644 |
| index 0000000..16252ee |
| --- /dev/null |
| +++ b/fixtures/docx/features.expected.json |
| @@ -0,0 +1 @@ |
| +{"alternate-body":{"same_semantics_as":"core"},"numbered-heading":{"block":"heading","level":1,"text":"Numbered heading","not":"list_item"},"link":{"label":"Reference","target":"https://example.test/reference","relationship_mode":"External"},"table":{"rows":[["Cell A","Cell B"],["Cell C","Cell D"]]},"image":{"media":"word/media/pixel.png","mime":"image/png","width_px":1,"height_px":1,"alt":"One green pixel"}} |
diff --git a/fixtures/docx/generate-update.mjs b/fixtures/docx/generate-update.mjs| new file mode 100644 |
| index 0000000..ba4cb07 |
| --- /dev/null |
| +++ b/fixtures/docx/generate-update.mjs |
| @@ -0,0 +1,16 @@ |
| +// Synthetic supported-subset projection, not a DOCX parser. Upstream's |
| +// ProseMirror/Yjs bridge authors bytes for the existing Store/export path. |
| +import { readFileSync, writeFileSync } from 'node:fs' |
| +import * as Y from 'yjs' |
| +import { Schema } from 'prosemirror-model' |
| +import { schema as basic } from 'prosemirror-schema-basic' |
| +import { addListNodes } from 'prosemirror-schema-list' |
| +import { prosemirrorToYXmlFragment } from 'y-prosemirror' |
| +const schema = new Schema({nodes: addListNodes(basic.spec.nodes, 'paragraph block*', 'block'), marks: basic.spec.marks}) |
| +const expected = JSON.parse(readFileSync(new URL('./core.expected.json', import.meta.url))) |
| +const node = schema.nodeFromJSON(expected) |
| +node.check() |
| +const doc = new Y.Doc({guid: 'docx-core-projection'}) |
| +doc.clientID = 8107 |
| +prosemirrorToYXmlFragment(node, doc.getXmlFragment('prosemirror')) |
| +writeFileSync(new URL('./core.update.bin', import.meta.url), Y.encodeStateAsUpdate(doc)) |
diff --git a/fixtures/docx/generate.py b/fixtures/docx/generate.py| new file mode 100644 |
| index 0000000..219efcd |
| --- /dev/null |
| +++ b/fixtures/docx/generate.py |
| @@ -0,0 +1,47 @@ |
| +"""Deterministic synthetic OPC fixtures; no document conversion code.""" |
| +import json |
| +from pathlib import Path |
| +import struct |
| +import zipfile |
| +import zlib |
| + |
| +ROOT = Path(__file__).parent |
| +W = 'http://schemas.openxmlformats.org/wordprocessingml/2006/main' |
| +R = 'http://schemas.openxmlformats.org/officeDocument/2006/relationships' |
| +P = 'http://schemas.openxmlformats.org/package/2006/relationships' |
| + |
| +def paragraph(text, properties=''): |
| + return '<w:p>' + properties + '<w:r><w:t xml:space="preserve">' + text + '</w:t></w:r></w:p>' |
| + |
| +core = '<w:p><w:pPr><w:pStyle w:val="Heading1"/></w:pPr><w:r><w:t>Section</w:t></w:r></w:p>' |
| +core += '<w:p><w:r><w:t xml:space="preserve">Plain </w:t></w:r><w:r><w:rPr><w:b/></w:rPr><w:t>bold</w:t></w:r><w:r><w:t xml:space="preserve"> </w:t></w:r><w:r><w:rPr><w:i/></w:rPr><w:t>italic</w:t></w:r></w:p>' |
| +for text, num, level in [('Outer',1,0),('Inner',1,1),('First',2,0),('Second',2,0)]: |
| + core += paragraph(text, f'<w:pPr><w:numPr><w:ilvl w:val="{level}"/><w:numId w:val="{num}"/></w:numPr></w:pPr>') |
| +numbered = paragraph('Numbered heading', '<w:pPr><w:pStyle w:val="Heading1"/><w:numPr><w:ilvl w:val="0"/><w:numId w:val="2"/></w:numPr></w:pPr>') |
| +link = '<w:p><w:hyperlink r:id="rLink"><w:r><w:t>Reference</w:t></w:r></w:hyperlink></w:p>' |
| +table = '<w:tbl><w:tblPr/><w:tblGrid><w:gridCol w:w="2400"/><w:gridCol w:w="2400"/></w:tblGrid>' + ''.join('<w:tr>'+''.join('<w:tc><w:tcPr/>'+paragraph(c)+'</w:tc>' for c in row)+'</w:tr>' for row in [('Cell A','Cell B'),('Cell C','Cell D')])+'</w:tbl>' |
| +image = '<w:p><w:r><w:drawing><wp:inline><wp:extent cx="9525" cy="9525"/><wp:docPr id="1" name="Synthetic pixel" descr="One green pixel"/><a:graphic><a:graphicData uri="http://schemas.openxmlformats.org/drawingml/2006/picture"><pic:pic><pic:nvPicPr><pic:cNvPr id="1" name="pixel.png"/><pic:cNvPicPr/></pic:nvPicPr><pic:blipFill><a:blip r:embed="rImage"/><a:stretch><a:fillRect/></a:stretch></pic:blipFill><pic:spPr><a:xfrm><a:off x="0" y="0"/><a:ext cx="9525" cy="9525"/></a:xfrm><a:prstGeom prst="rect"><a:avLst/></a:prstGeom></pic:spPr></pic:pic></a:graphicData></a:graphic></wp:inline></w:drawing></w:r></w:p>' |
| +numbering = f'<w:numbering xmlns:w="{W}">' + ''.join(f'<w:abstractNum w:abstractNumId="{n}">'+''.join(f'<w:lvl w:ilvl="{level}"><w:start w:val="1"/><w:numFmt w:val="{kind}"/><w:lvlText w:val="{mark}"/></w:lvl>' for level in range(2))+f'</w:abstractNum><w:num w:numId="{n}"><w:abstractNumId w:val="{n}"/></w:num>' for n,kind,mark in [(1,'bullet','•'),(2,'decimal','%1.')])+'</w:numbering>' |
| +styles=f'<w:styles xmlns:w="{W}"><w:style w:type="paragraph" w:styleId="Heading1"><w:name w:val="heading 1"/><w:pPr><w:outlineLvl w:val="0"/></w:pPr></w:style></w:styles>' |
| +def chunk(kind, data): |
| + return struct.pack('>I',len(data))+kind+data+struct.pack('>I',zlib.crc32(kind+data)&0xffffffff) |
| +png=b'\x89PNG\r\n\x1a\n'+chunk(b'IHDR',struct.pack('>IIBBBBB',1,1,8,2,0,0,0))+chunk(b'IDAT',zlib.compress(b'\x00\x00\xff\x00'))+chunk(b'IEND',b'') |
| +manifest=[] |
| +for name,body,part in [('core',core,'word/document.xml'),('alternate-body',core,'word/document2.xml'),('numbered-heading',numbered,'word/document.xml'),('link',link,'word/document.xml'),('table',table,'word/document.xml'),('image',image,'word/document.xml')]: |
| + rels=[('rStyles','styles','styles.xml',''),('rNumbering','numbering','numbering.xml','')] |
| + if name=='link': rels.append(('rLink','hyperlink','https://example.test/reference',' TargetMode="External"')) |
| + if name=='image': rels.append(('rImage','image','media/pixel.png','')) |
| + files={ |
| + '[Content_Types].xml':'<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Default Extension="xml" ContentType="application/xml"/><Default Extension="png" ContentType="image/png"/>'+f'<Override PartName="/{part}" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/><Override PartName="/word/styles.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.styles+xml"/><Override PartName="/word/numbering.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.numbering+xml"/></Types>', |
| + '_rels/.rels':f'<Relationships xmlns="{P}"><Relationship Id="root" Type="{R}/officeDocument" Target="{part}"/></Relationships>', |
| + part:f'<w:document xmlns:w="{W}" xmlns:r="{R}" xmlns:wp="http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing" xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:pic="http://schemas.openxmlformats.org/drawingml/2006/picture"><w:body>{body}<w:sectPr/></w:body></w:document>', |
| + 'word/_rels/'+Path(part).name+'.rels':f'<Relationships xmlns="{P}">'+''.join(f'<Relationship Id="{i}" Type="{R}/{kind}" Target="{target}"{mode}/>' for i,kind,target,mode in rels)+'</Relationships>', |
| + 'word/styles.xml':styles,'word/numbering.xml':numbering, |
| + } |
| + if name=='image': files['word/media/pixel.png']=png |
| + with zipfile.ZipFile(ROOT/(name+'.docx'),'w',compression=zipfile.ZIP_STORED) as archive: |
| + for path,data in sorted(files.items()): |
| + info=zipfile.ZipInfo(path,date_time=(2026,1,1,0,0,0)); info.external_attr=0o100644<<16 |
| + archive.writestr(info,data) |
| + manifest.append({'file':name+'.docx','body_part':part}) |
| +(ROOT/'manifest.json').write_text(json.dumps(manifest,indent=2)+'\n') |
diff --git a/fixtures/docx/image.docx b/fixtures/docx/image.docx| new file mode 100644 |
| index 0000000..7c42904 |
| Binary files /dev/null and b/fixtures/docx/image.docx differ |
diff --git a/fixtures/docx/link.docx b/fixtures/docx/link.docx| new file mode 100644 |
| index 0000000..00dd7dc |
| Binary files /dev/null and b/fixtures/docx/link.docx differ |
diff --git a/fixtures/docx/manifest.json b/fixtures/docx/manifest.json| new file mode 100644 |
| index 0000000..53f4c42 |
| --- /dev/null |
| +++ b/fixtures/docx/manifest.json |
| @@ -0,0 +1,26 @@ |
| +[ |
| + { |
| + "file": "core.docx", |
| + "body_part": "word/document.xml" |
| + }, |
| + { |
| + "file": "alternate-body.docx", |
| + "body_part": "word/document2.xml" |
| + }, |
| + { |
| + "file": "numbered-heading.docx", |
| + "body_part": "word/document.xml" |
| + }, |
| + { |
| + "file": "link.docx", |
| + "body_part": "word/document.xml" |
| + }, |
| + { |
| + "file": "table.docx", |
| + "body_part": "word/document.xml" |
| + }, |
| + { |
| + "file": "image.docx", |
| + "body_part": "word/document.xml" |
| + } |
| +] |
diff --git a/fixtures/docx/numbered-heading.docx b/fixtures/docx/numbered-heading.docx| new file mode 100644 |
| index 0000000..fbd0893 |
| Binary files /dev/null and b/fixtures/docx/numbered-heading.docx differ |
diff --git a/fixtures/docx/table.docx b/fixtures/docx/table.docx| new file mode 100644 |
| index 0000000..f05cc7f |
| Binary files /dev/null and b/fixtures/docx/table.docx differ |
diff --git a/internal/docs/docx_projection_test.go b/internal/docs/docx_projection_test.go| new file mode 100644 |
| index 0000000..21c2611 |
| --- /dev/null |
| +++ b/internal/docs/docx_projection_test.go |
| @@ -0,0 +1,49 @@ |
| +package docs |
| + |
| +import ( |
| + "bytes" |
| + "context" |
| + "net/http" |
| + "os" |
| + "strings" |
| + "testing" |
| +) |
| + |
| +func TestDOCXCoreProjectionThroughStoredYjsAndExportRoutes(t *testing.T) { |
| + app, err := NewForTest(t) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + owner := app.TestPerson(t, "docx-fixture@example.test") |
| + doc, err := app.Store().Create(context.Background(), owner, "DOCX fixture") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + update := readFixture(t, "../../fixtures/docx/core.update.bin") |
| + if _, err := app.Store().ApplyUpdate(context.Background(), doc, app.TestActorFor(t, doc), update); err != nil { |
| + t.Fatal(err) |
| + } |
| + expected, err := os.ReadFile("../../fixtures/docx/core.expected.md") |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + markdown := serve(app, app.TestRequest(t, "docx-fixture@example.test", http.MethodGet, "/d/"+doc.Sqid+"/export.md", nil)) |
| + if markdown.Code != 200 { |
| + t.Fatalf("Markdown route status %d", markdown.Code) |
| + } |
| + want := "# DOCX fixture\n\n" + string(expected) |
| + if markdown.Body.String() != want { |
| + t.Fatalf("persisted Yjs projection changed structure\ngot %q\nwant %q", markdown.Body.String(), want) |
| + } |
| + pdf := serve(app, app.TestRequest(t, "docx-fixture@example.test", http.MethodGet, "/d/"+doc.Sqid+"/export.pdf", nil)) |
| + if pdf.Code != 200 || !bytes.HasPrefix(pdf.Body.Bytes(), []byte("%PDF-")) { |
| + t.Fatal("PDF route did not emit a PDF") |
| + } |
| + if !strings.HasPrefix(pdf.Header().Get("Content-Type"), "application/pdf") { |
| + t.Fatal("PDF content type missing") |
| + } |
| + denied := serve(app, app.TestRequest(t, "foreign@example.test", http.MethodGet, "/d/"+doc.Sqid+"/export.md", nil)) |
| + if denied.Code != 404 { |
| + t.Fatal("corpus document escaped owner boundary") |
| + } |
| +} |
diff --git a/internal/export/docx_corpus_test.go b/internal/export/docx_corpus_test.go| new file mode 100644 |
| index 0000000..47ff9ed |
| --- /dev/null |
| +++ b/internal/export/docx_corpus_test.go |
| @@ -0,0 +1,192 @@ |
| +package export_test |
| + |
| +import ( |
| + "archive/zip" |
| + "bytes" |
| + "encoding/json" |
| + "encoding/xml" |
| + "image/png" |
| + "io" |
| + "os" |
| + "path" |
| + "strings" |
| + "testing" |
| + |
| + "github.com/carlosframework/docs/internal/export" |
| +) |
| + |
| +const corpusDirectory = "../../fixtures/docx/" |
| + |
| +func corpusFile(t *testing.T, name string) []byte { |
| + t.Helper() |
| + data, err := os.ReadFile(corpusDirectory + name) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + return data |
| +} |
| + |
| +func docxParts(t *testing.T, name string) map[string][]byte { |
| + t.Helper() |
| + raw := corpusFile(t, name) |
| + archive, err := zip.NewReader(bytes.NewReader(raw), int64(len(raw))) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + parts := map[string][]byte{} |
| + for _, file := range archive.File { |
| + if path.Clean(file.Name) != file.Name || strings.HasPrefix(file.Name, "/") || strings.HasPrefix(file.Name, "../") { |
| + t.Fatal("fixture contains unsafe OPC part") |
| + } |
| + if _, exists := parts[file.Name]; exists { |
| + t.Fatal("duplicate OPC part") |
| + } |
| + reader, err := file.Open() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + data, err := io.ReadAll(reader) |
| + reader.Close() |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + parts[file.Name] = data |
| + if strings.HasSuffix(file.Name, ".xml") || strings.HasSuffix(file.Name, ".rels") { |
| + decoder := xml.NewDecoder(bytes.NewReader(data)) |
| + for { |
| + _, err := decoder.Token() |
| + if err == io.EOF { |
| + break |
| + } |
| + if err != nil { |
| + t.Fatalf("malformed fixture XML %s", file.Name) |
| + } |
| + } |
| + } |
| + } |
| + return parts |
| +} |
| + |
| +func TestDOCXCorpusPackageRelationshipsAndFeatureExpectations(t *testing.T) { |
| + var manifest []struct { |
| + File string `json:"file"` |
| + BodyPart string `json:"body_part"` |
| + } |
| + if err := json.Unmarshal(corpusFile(t, "manifest.json"), &manifest); err != nil { |
| + t.Fatal(err) |
| + } |
| + if len(manifest) != 6 { |
| + t.Fatal("fixture corpus changed without expectation review") |
| + } |
| + for _, item := range manifest { |
| + t.Run(item.File, func(t *testing.T) { |
| + parts := docxParts(t, item.File) |
| + var roots struct { |
| + Relationships []struct { |
| + Type string `xml:"Type,attr"` |
| + Target string `xml:"Target,attr"` |
| + } `xml:"Relationship"` |
| + } |
| + if err := xml.Unmarshal(parts["_rels/.rels"], &roots); err != nil { |
| + t.Fatal(err) |
| + } |
| + body := "" |
| + for _, rel := range roots.Relationships { |
| + if strings.HasSuffix(rel.Type, "/officeDocument") { |
| + body = rel.Target |
| + } |
| + } |
| + if body != item.BodyPart || len(parts[body]) == 0 { |
| + t.Fatal("officeDocument relationship does not select expected body") |
| + } |
| + // XML attributes need explicit tags; these independent reads validate the |
| + // archived wire shape, not the Python fixture generator's in-memory data. |
| + var rels struct { |
| + Relationships []struct { |
| + ID string `xml:"Id,attr"` |
| + Target string `xml:"Target,attr"` |
| + Mode string `xml:"TargetMode,attr"` |
| + } `xml:"Relationship"` |
| + } |
| + relPart := path.Join(path.Dir(body), "_rels", path.Base(body)+".rels") |
| + if err := xml.Unmarshal(parts[relPart], &rels); err != nil { |
| + t.Fatal(err) |
| + } |
| + for _, rel := range rels.Relationships { |
| + if rel.Mode != "External" && len(parts[path.Join(path.Dir(body), rel.Target)]) == 0 { |
| + t.Fatal("internal relationship target missing") |
| + } |
| + } |
| + content := string(parts[body]) |
| + switch item.File { |
| + case "core.docx", "alternate-body.docx": |
| + for _, text := range []string{"Section", "Plain ", "bold", "italic", "Outer", "Inner", "First", "Second", `<w:ilvl w:val="1"/>`} { |
| + if !strings.Contains(content, text) { |
| + t.Fatalf("core source lacks %q", text) |
| + } |
| + } |
| + if item.File == "alternate-body.docx" && parts["word/document.xml"] != nil { |
| + t.Fatal("alternate-body fixture accidentally permits fixed-path parsing") |
| + } |
| + case "numbered-heading.docx": |
| + if !strings.Contains(content, `w:pStyle w:val="Heading1"`) || !strings.Contains(content, "w:numPr") || !strings.Contains(content, "Numbered heading") { |
| + t.Fatal("numbered heading no longer combines both properties") |
| + } |
| + case "link.docx": |
| + found := false |
| + for _, rel := range rels.Relationships { |
| + if rel.ID == "rLink" && rel.Mode == "External" && rel.Target == "https://example.test/reference" { |
| + found = true |
| + } |
| + } |
| + if !found || !strings.Contains(content, `w:hyperlink r:id="rLink"`) || !strings.Contains(content, "Reference") { |
| + t.Fatal("link source or relationship missing") |
| + } |
| + case "table.docx": |
| + if strings.Count(content, "<w:tr>") != 2 || strings.Count(content, "<w:tc>") != 4 { |
| + t.Fatal("table geometry changed") |
| + } |
| + for _, text := range []string{"Cell A", "Cell B", "Cell C", "Cell D"} { |
| + if !strings.Contains(content, text) { |
| + t.Fatal("table cell missing") |
| + } |
| + } |
| + case "image.docx": |
| + config, err := png.DecodeConfig(bytes.NewReader(parts["word/media/pixel.png"])) |
| + if err != nil || config.Width != 1 || config.Height != 1 { |
| + t.Fatal("synthetic embedded image invalid") |
| + } |
| + if !strings.Contains(content, `r:embed="rImage"`) || !strings.Contains(content, `descr="One green pixel"`) { |
| + t.Fatal("image reference or alt text missing") |
| + } |
| + } |
| + }) |
| + } |
| +} |
| + |
| +func TestDOCXCoreExpectedProjectionSurvivesExistingExports(t *testing.T) { |
| + doc, err := export.Parse("", string(corpusFile(t, "core.expected.json"))) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if got, want := export.Markdown(doc), string(corpusFile(t, "core.expected.md")); got != want { |
| + t.Fatalf("structural Markdown projection mismatch\ngot: %q\nwant: %q", got, want) |
| + } |
| + data, report, err := export.PDF(doc) |
| + if err != nil { |
| + t.Fatal(err) |
| + } |
| + if !report.Empty() { |
| + t.Fatal("unexpected core projection substitution") |
| + } |
| + pdf := readPDF(t, data) |
| + extracted := "" |
| + for _, page := range pdf.pages(t) { |
| + extracted += pdf.text(t, page) |
| + } |
| + for _, text := range []string{"Section", "Plain", "bold", "italic", "Outer", "Inner", "First", "Second"} { |
| + if !strings.Contains(extracted, text) { |
| + t.Fatalf("PDF projection lost %q", text) |
| + } |
| + } |
| +} |