Add adversarial XSS corpus test for the HTML cleaner (initial) (#3627)
- Add adversarial XSS corpus test for LJ::CleanHTML
An initial improvement to the cleaner test suite: a library of known XSS
vectors (classic, HTML5, scheme-obfuscation, CSS, and mutation-XSS) run
through the real clean_event / clean_comment / clean_userbio display paths,
asserting no JavaScript-execution vector survives. The oracle checks whether
an exec vector remains rather than matching exact output, so it stays
meaningful as cleaner internals change.
Baselines what the current cleaner catches (the whole standard XSS canon --
177 assertions pass) versus what it misses: clean_event's permissive
allow-most mode lets SVG/MathML foreign content and its SMIL/XML-Events
children through with only attribute-level filtering, whereas clean_comment's
strict allowlist strips them. Those known gaps are marked TODO so the suite
stays green while recording exactly what is missed.
More coverage to follow (other surfaces/formatting modes, non-XSS viewer
harms, and a stronger test oracle).
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01JhHtJhPnKr7zaCHZsfAd1P
- cleaner-xss: make the safety oracle browser-accurate; drop redundant clean
Address review feedback on the is_unsafe oracle:
- Decode entities single-pass like a browser (decode_entities_once): raw
/
/: and numeric entities (hex any-case incl. A-F and
uppercase X, plus decimal) are now decoded before scheme detection, so a
cleaner regression that emitted an entity-obfuscated javascript: URL can no
longer pass silently. An already-escaped 	 stays literal (inert) and
is not re-decoded, so it does not false-positive. Whitespace and control
chars are stripped the way browsers ignore them in URLs/schemes.
- Add oracle self-checks asserting it sees obfuscated schemes/handlers and
ignores inert escaped input and attribute-value text.
- Clean each payload once and reuse it for both the assertion and the diag
instead of cleaning twice.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01JhHtJhPnKr7zaCHZsfAd1P
- cleaner-xss: cover all surfaces and formatting modes
Expand the corpus from clean_event/clean_comment/clean_userbio into a
surface x mode matrix: entries and comments in each formatting mode
(default html, preformatted, markdown, syndicated feed HTML), plus subjects
(clean_subject / clean_subject_all / clean_and_trim_subject). Adds a
markdown-only vector set for what Text::Markdown can generate (link/image
destinations, reference links, autolinks, raw HTML passthrough, code spans)
before the clean runs.
No new leaks: the current cleaner neutralizes the whole corpus across every
mode and surface -- the modes share the security-critical allow/eat/remove
lists, and subjects strip to their tiny allowlist. The one documented gap
(SVG/MathML foreign content in clean_event) is unchanged and still TODO.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01JhHtJhPnKr7zaCHZsfAd1P
- Add characterization tests for third-party resource loading in the cleaner
Document (rather than change) how the cleaner treats external resource loading
in user content. Loading an external image or inline CSS url() leaks the
viewer's IP/User-Agent to that host -- a privacy tradeoff DW intentionally
accepts (hotlinking is a long-standing feature; DW::Proxy exists only to fix
http->https mixed content, not to hide the viewer).
These tests pin that accepted behavior so any change is deliberate, and lock in
the controls that stop the dangerous variants: blocks are removed (so
there's no place for the attribute selectors that CSS-based exfiltration needs)
and script-y CSS (url(javascript:), expression()) is stripped from inline
styles. No behavior change.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01JhHtJhPnKr7zaCHZsfAd1P
- cleaner-xss: address review + add clean_embed coverage
Review fixes:
- Retarget the SVG/MathML TODO gaps to the dangerous constructs that actually
survive (SMIL / driving handlers/navigation, XML Events
, MathML foreign-HTML), instead of asserting "strip
all svg/math" -- which a namespace-aware sanitizer wouldn't do anyway.
- Add positive controls (benign inline formatting, a safe link, plain text are
preserved) so a regression that strips everything can't pass as vacuously safe.
- Scope the CSS checks (expression/@import/binding/url(js)) to style="" values
and blocks so benign body text can't false-positive.
New coverage:
- clean_embed (the RTE media-embed path) with a media-permitting oracle
(allow_media => 1):
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
github posting in ![[site community profile]](https://www.dreamwidth.org/img/comm_staff.png){kind=small}
changelog
