Index

How an MX Audit Proves Itself

· 6 min read

Audit reports usually carry their own credibility problem. The reader has the document. They do not have the evidence that produced it. They can ask the writer; the writer can show their working; the working may or may not match what shipped. An MX audit closes that gap from the other side. An auditor reading an MX deliverable can walk from the document to every signal it depends on, without needing our word for anything.

Evidence rides inside the PDF

Every PDF an MX audit produces carries a structured XMP packet inside its metadata stream. The packet declares the artefact's conformance claim: pdfuaid:Part=1, the Level 2 declaration for ISO 14289-1, the tagged-PDF standard that EAA Directive 2019/882 leans on. It also carries a set of MX-namespaced identity fields populated from the source frontmatter: who authored it, when, what content type it is, which audiences it is for, what it conforms to. The same packet embeds the full AI provenance evidence chain inline under XMP-mx:ProvenanceAiPayload: every step that touched the artefact, who or what executed each one, what inputs went in, what verdict came out, and which AI-governance frameworks the step is recorded against (EU AI Act, UK ICO AI guidance, NIST AI RMF, Colorado AI Act).

The same evidence chain also lives in a sidecar JSON file adjacent to the PDF on disk (<basename>.provenance.ai.json) along with its deterministic companion (<basename>.provenance.deterministic.json) for the rule-driven steps. The on-disk copies serve EAA accessibility-conformance evidence and any inspector who prefers file-system access. The inline XMP copy serves the inspector who only has the PDF.

To extract the embedded chain on any MX PDF: exiftool -b -XMP-mx:ProvenanceAiPayload <file>.pdf. The -b flag matters; without it exiftool prepends a label and the JSON does not parse.

Reviewer signals stay separate from the client surface

Every audit goes through a battery of deterministic gates before delivery. They catch scope drift, hyperbole in prose, claims a numeric verifier could not corroborate against the source data, voice inconsistencies. Each gate that flags something records the finding to a structured store (audit_errors.json) and the pipeline continues to a deliverable PDF. This is the always-produce-PDF contract: a failing gate raises a warning, not an exit code, so the client receives a document that names its own gaps rather than a silent failure.

Those reviewer-facing findings live in a sibling sidecar, not inside the client-facing report. Each delivery ships with a companion markdown file (<basename>-report-findings.md) carrying every finding the gates raised, by severity, with the gate name, the category, the timestamp, and the supporting evidence. The reviewer reads this before signing off; the client reads the report; the auditor reads either or both.

The separation is structural, not cosmetic. The findings sidecar carries its own MX frontmatter declaring contentType: audit-findings and a companion pointer to the report. The report's provenance chain records the reviewer's sign-off statement (if any) as an audit step, so the evidence chain still names the human who released the document. The chain integrity is intact; the surface that meets the client is clean.

One scoring vocabulary, machine-verifiable

The Balanced Scorecard speaks one scoring vocabulary across every cell. The audited row reports a band (Excellent, Good, Could Be Better, Needs Improvement) with a letter grade alongside (A through D). The peer-comparison cell uses the same band, never a raw median. Each cell reads as A (median), B (median), C (median), or D (median). An auditor scanning the scorecard sees site versus cohort in a single eye-pass.

The change is also useful when the scorecard is consumed by a machine. The bands map cleanly to a four-value enum; a tooling pipeline can compare site to peer without parsing or normalising. The methodology that maps a numeric score to a band lives in mx-reginald/audit/lib/scoring-methodology.json, the single source of truth for every score's inputs, weights, and band boundaries. The auditor can verify any reported band by re-running the methodology against the raw figures in audit_averages.json; the figures are committed alongside the delivery in the .infill/ archive.

A regression test that proves the claim on every render

"MX Compatible" is a regression-tested contract, not a marketing line. tests/test-pdf-mx-compatible.js renders a small fixture markdown through the same PDF pipeline that produces client deliveries, then asserts every signal an inspector relies on:

  • The AI provenance sidecar exists adjacent to the PDF.
  • The deterministic provenance sidecar exists adjacent to the PDF.
  • The PDF XMP packet declares pdfuaid:Part=1.
  • The MX identity fields in XMP mirror the source frontmatter byte-for-byte.
  • The provenance pointer fields name the adjacent sidecars by basename.
  • The full AI evidence chain is embedded inline under XMP-mx:ProvenanceAiPayload.
  • The inline copy matches the adjacent .ai.json after JSON normalisation.

The test is wired into npm test. Any change that would silently break the MX Compatible contract breaks the test instead. The pipeline does not ship a PDF that fails this test, and the operator who runs the pipeline gets a clear, named failure pointing at the missing signal.

What an auditor can do with this

An MX audit report can be verified without leaving the document. exiftool -b -XMP-mx:ProvenanceAiPayload <file>.pdf | jq . returns the full AI chain on the command line. The deterministic companion is named in the same packet under XMP-mx:ProvenanceCompanion; an inspector with file-system access can resolve and read it. The reviewer's findings live alongside in the findings sidecar. The methodology that produced every score lives at scoring-methodology.json in the audit toolkit, public and re-runnable. The frameworks each provenance step is recorded against are named in the chain itself, so an auditor working a regulatory clause can walk from the clause to the steps the report cites.

An auditor who prefers a browser to a terminal can use the interactive inspector at /tools/pdf-inspector.html. The inspector accepts any MX PDF (the QR code on the cover badge resolves to the same URL) and walks the full evidence chain end-to-end in the browser. A reader who wants the short version of what each signal means and which standard it cites can read the explainer at /learn/mx-for-pdfs.html.

None of this constitutes a compliance claim against any specific regulation. EAA conformance, EU AI Act conformance, ICO accountability: all of those remain legal duties of the publishing organisation. What MX provides is the evidence vehicle: the structured, tamper-evident, machine-readable trail that the organisation needs to produce when asked. An auditor walking that trail has the cover, the inspector, the explainer, the sidecars, the methodology, and the framework citations to lean on. That is the contract.

Back to Top