Incomplete <!DOCTYPE> is not parsed correctly, drops the rest of the input

#40727112 Filed 2020-10-22 HTML parser / tokenizer

Live reproducer

Each snippet below is parsed by your browser in an isolated <iframe srcdoc>. The script inspects the resulting DOM and compares it against the spec-correct result (what Firefox produces). On an affected Chrome the last snippet before end-of-input is silently truncated.

Running...
InputYour browserExpected (spec)

Reload the page to re-run. The result reflects the Chrome build you are viewing this page with.

What goes wrong

Reported behavior

<!DOC> as a whole document produces an empty document: no documentElement, no children.

Abc<!d>Hi loses the trailing Hi: only Abc survives.

Root cause

When the tokenizer is in the markup declaration open state (after <!) and sees d/D, it looks ahead for the 7-character string DOCTYPE. The look-ahead has three results: match, no-match, or kNotEnoughCharacters.

// html_tokenizer.cc, kMarkupDeclarationOpenState
} else if (cc == 'D' || cc == 'd') {
  auto result = source.LookAheadIgnoringCase(html_tokenizer_names::kDoctype);
  if (result == kDidMatch) { ... switch to kDOCTYPEState ... }
  else if (result == kNotEnoughCharacters)
    return HaveBufferedCharacterToken();   // wait for more input
}

SegmentedString::LookAheadSlowCase() returns kNotEnoughCharacters whenever fewer than 7 characters remain -- even after the stream is closed. For <!DOC> only DOC> (plus the EOF marker) remains, which is shorter than DOCTYPE, so the tokenizer parks waiting for bytes that will never arrive. The bogus comment and everything after it are dropped.

// segmented_string.h, before
if (count > length())
  return kNotEnoughCharacters;   // even when the stream is already closed

Links

Chromium issue 40727112