惯性聚合 高效追踪和阅读你感兴趣的博客、新闻、科技资讯
阅读原文 在惯性聚合中打开

推荐订阅源

Spread Privacy
Spread Privacy
K
Kaspersky official blog
Application and Cybersecurity Blog
Application and Cybersecurity Blog
Forbes - Security
Forbes - Security
Hacker News - Newest:
Hacker News - Newest: "LLM"
The Last Watchdog
The Last Watchdog
SecWiki News
SecWiki News
Attack and Defense Labs
Attack and Defense Labs
Google DeepMind News
Google DeepMind News
Security Archives - TechRepublic
Security Archives - TechRepublic
S
Secure Thoughts
WordPress大学
WordPress大学
Microsoft Security Blog
Microsoft Security Blog
P
Proofpoint News Feed
云风的 BLOG
云风的 BLOG
V
Visual Studio Blog
Security Latest
Security Latest
TaoSecurity Blog
TaoSecurity Blog
Cyberwarzone
Cyberwarzone
S
SegmentFault 最新的问题
Cloudbric
Cloudbric
aimingoo的专栏
aimingoo的专栏
S
Schneier on Security
N
Netflix TechBlog - Medium
MyScale Blog
MyScale Blog
T
The Blog of Author Tim Ferriss
H
Hacker News: Front Page
C
Cybersecurity and Infrastructure Security Agency CISA
小众软件
小众软件
OSCHINA 社区最新新闻
OSCHINA 社区最新新闻
AWS News Blog
AWS News Blog
AI
AI
G
GRAHAM CLULEY
IT之家
IT之家
P
Privacy & Cybersecurity Law Blog
L
Lohrmann on Cybersecurity
Last Week in AI
Last Week in AI
D
Docker
Recent Announcements
Recent Announcements
O
OpenAI News
T
Threat Research - Cisco Blogs
GbyAI
GbyAI
S
Security @ Cisco Blogs
T
Troy Hunt's Blog
C
Check Point Blog
博客园 - 三生石上(FineUI控件)
A
About on SuperTechFans
The Cloudflare Blog
阮一峰的网络日志
阮一峰的网络日志
N
News and Events Feed by Topic

Henri Sivonen’s pages

Parin vuoden tutkimattomuus crates.io: Rust Package Registry Asiakirjatonta toimintaa It’s not wrong that "🤦🏼‍♂️".length == 7 Koulutartuntojen tilastointimenettely Perusteasiakirjoja hallussapitämättä ikärajoitettu Asiantuntijat ja nukkuva vallan vahtikoira Koronapassilausunto Suppealla tietopohjalla ohimeneväksi väitetty Text Encoding Menu in 2021 The Text Encoding Submenu Is Gone An HTML5 Conformance Checker Not Part of the Technology Stack Browser Technology Stack Bogo-XML Declaration Returns to Gecko A Look at Encoding Detection and Encoding Menu Telemetry from Firefox 86 Why Supporting Unlabeled UTF-8 in HTML on the Web Would Be Problematic Rust Target Names Aren’t Passed to LLVM Toimintamalli Activating Browser Modes with Doctype Johtopäätöksiä mallin rakenteesta Tehtävänmäärittelyä kirjoittamatta ja kuolemia laskematta laumasuojamallinnettu Character Encoding Menu in 2014 Erillissuosituksen tarpeettomuudesta yleissuosituksen poikkeukseksi? STM:n maskiaikajana Rust 2021 Oma-aloitteisesti mallinnettu Kokopinovaatimuksin kilpailutettu chardetng: A More Compact Character Encoding Detector for the Legacy Web Varauksia paisutellen tiedotettu Perusteasiakirjoitta tiedotettu Always Use UTF-8 & Always Label Your HTML Saying So IME Smoke Testing The Validator.nu HTML Parser About the Hiragino Fonts with CSS It’s Time to Stop Adding New Features for Non-Unicode Execution Encodings in C++ Rust 2020 The Last of the Parsing Quirks About about:blank Rust 2019 a Web-Compatible Character Encoding Library in Rust How I Wrote a Modern C++ Library in Rust Using cargo-fuzz to Transfer Code Review of Simple Safe Code to Complex Code that Uses unsafe A Rust Crate that Also Quacks Like a Modern C++ Library #Rust2018 No Namespaces in JSON, Please A Lecture about HTML5 Julkisesti luotettu varmenne ikidomainille TLS:ää (SSL:ää) varten -webkit-HTML5 Lists in Attribute Values The Sad Story of PNG Gamma “Correction” If You Want Software Freedom on Phones, You Should Work on Firefox OS, Custom Hardware and Web App Self-Hostablility HTML5 Parser Improvements ARIA in HTML5 Integration: Document Conformance (Draft, Take Two) Schema.org and Pre-Existing Communities Lowering memory requirements by replacing Schematron HTML5 Parsing in Gecko: A Build Introducing SAX Tree NVDL Support in Validator.nu HOWTO Avoid Being Called a Bozo When Producing XML An Unofficial Q&A about the Discontinuation of the XHTML2 WG Thoughts on HTML5 Becoming a W3C Recommendation Four Finnish Banks Training Users to Give Banking Credentials to Another Site Unimpressed by Leopard Sergeant Semantics The Content Sink Inheritance Diagram – 2006-06-30 What is EME? About Points and Pixels as Units The Performance Cost of the HTML Tree Builder Social Media Impression Management The spacer Element Is Gone Openmind 2006 Performance Mistake XHTML and Mobile Devices WebM-Enabled Browser Usage Share Exceeds H.264-Enabled Browser Usage Share on Desktop (in StatCounter Numbers) Vendor Prefixes Are Hurting the Web Accept-Charset Is No More Dualroids Writing Structural Stylable Document in Mozilla Editor ISO-8859-15 on haitallinen Hourglass The Scientific Method According to Hixie Maemo Source Code Karpelan lukkovertaus ontuu Digitaalisesta arkistoinnista ARIA in HTML5 Integration: Document Conformance (Draft) XHTML—What’s the Point? (Draft, incomplete) Mac OS X Browser Comparison HOWTO Spot a Wannabe Web Standards Advocate An Idea About Intermediate Language Trees and Web UI Generation Thoughts on Using SSL/TLS Certificates as the Solution to Phishing Bureaucracy Meets the Web Europe Day HOWTO Establish a 100% Literacy Rate What to Do with All These Photos? Charmod Norm Checking Validator Web Service Interface Ideas DTDs Don’t Work on the Web EFFI’s Day in Court Speaking at XTech
HTML5 Parser-Based View Source Syntax Highlighting
Henri Sivonen · 2012-03-15 · via Henri Sivonen’s pages

A new implementation of the View Source HTML and XML syntax highlighting has landed in Firefox.

Why?

The reason there is a new implementation is that the old implemention was based on the old HTML parser that we want to get rid of. The old View Source implementation was standing in the way of the goal to remove the old parser. Also, the old parser did some incorrect highlighting. Most notably, it flagged the unnecessary-but-permitted slash as an error on void elements (e.g. <br/>) because all such slashes were bogus in non-X HTML prior to HTML5.

The reason why the new implementation uses the HTML5 parser instead of using something new and Orion-integrated in the dev tools is that the new implementation was written before there were publicized plans to integrate dev tools with View Source. Furthermore, there is no way to get HTML syntax highlighting right without the highlighter running the whole HTML(5) parsing algorithm, because tokenizer state transition decisions depend on the tree builder state.

New Features

The first and foremost feature is not user-visible per se. It is the non-use of the old parser code in order to be able to get rid of the old parser. However, using the old parser had user-visible consquences.

More Correct Highlighting

As already mentioned, the old parser unconditionally highlighted the slash in <foo/> as red regardless of the element name. Furthermore, the old parser failed to get the highlighting of tricky inline scripts right (when the inline script contained the string </script>). Highlighting of SVG and MathML content in text/html was wrong, too, since the old parser knew nothing about foreign content in text/html.

Consider the following highlighting by the old parser:

<!DOCTYPE html>
<html>
<head>
<title>Title</title>
<script>
var lt = "<";
<!--
var s = "<script>foo</script>";
-->
</script>
<style>
/* </foo> */
</style>
</head>
<body>
<p>Entity: &amp; </p>
<iframe><img></iframe>
<noscript><p>Not para</p></noscript>
<svg>
<title><![CDATA[bar]]></title>
<script><!-- this is a comment --></script>
</svg>
</body>
</html>

The first occurrence of </script> is highlighted as an end tag. The content of the SVG title and script elements is treated as if the elements were HTML elements of the same name.

Compare the above to the highlighting performed by the new implementation:

<!DOCTYPE html>
<html>
<head>
<title>Title</title>
<script>
var lt = "<";
<!--
var s = "<script>foo</script>";
-->
</script>
<style>
/* </foo> */
</style>
</head>
<body>
<p>Entity: &amp; </p>
<iframe><img></iframe>
<noscript><p>Not para</p></noscript>
<svg>
<title><![CDATA[bar]]></title>
<script></script>
</svg>
</body>
</html>

The HTML script is tokenized according to the HTML rules. Note that <----> inside an HTML script is not a comment node! In the SVG subtree, title and script are not special and can have CDATA sections or comments inside them. (The coloring of the HTML script end tag is inconsistent with other end tags, though, due to technical difficulties.)

Better Error Reporting

The old parser highlighted errors so rarely that it was easy to think it was not doing it at all. However, it did indeed have support for highlighting a couple of errors. I am aware of it highlighting doctypes that used XML syntax inappropriate for HTML and highlighting the already mentioned XML-style slash in tags.

To get feature parity with the old implementation, the new implementation had to support at least highlighting the XML-style slash when the use of the slash is wrong per HTML5 / HTML Living Standard. Since highlighting the slash correctly is among the more difficult error highlights and the Java version of the parser (from which the C++ version is mechanically translated) already supported full error reporting since it was originally written for a validator, I thought I could add support for the easier error highlights, too, while at it.

But why stop at highlights without explaining them? I also made the parser attach error messages to the highlights as tooltips. (Unfortunately, Firefox has long-standing accessibility problems with tooltips, so the error messages are not keyboard-accessible at the moment.)

The new View Source implementation produces results like this (note the tooltips):

<!DOCTYPE HTML PUBLIC"-//W3C//DTD HTML 4.01//EN">
<form>
<table>
<h1>Error test</h1>
<tr><td><div>cell<td>another cell
</table>
<form>
<select><select>
<div>
<p class="foo"id="bar">
<p/>
<br/>
<h2><h3></h3></h2>

<svg>
<![CDATA[this is text]]>
<div>



<p><i><b>bold italic</i></b></p>
<p>&#x0000;</p>
<p>&auml </p>
<p>&foo </p>
<a class=foohref="foo">
<a></a>
</a>

Note: The tooltips will have line breaks between multiple error messages in one tooltip when viewed in the View Source window in Firefox. The lack of line breaks in Firefox in other contexts (including this page) is a known HTML5 violation bug.

Off-The-Main-Thread Highlighting

As a consequence of the off-the-main-thread design of the HTML5 parser in Firefox, the highlight computation now happens off the main thread.

Limitations

The above may set expectations too high, so it is important to lower them right away.

This Is Not a Validator!

All the errors you have seen above are parse errors. Parse errors are errors defined as such by the HTML parsing algorithm. There is much more to checking HTML validity than just finding the parse errors.

For example, putting a div element as a child of an ul element or as a child of a span element is not a parse error. In an HTML validator, content model errors like that are detected by a validation layer above the parser. The View Source implementation in Firefox does not have a validation layer at all.

The lack of a validation layer has counter-intuitive consequences. The HTML parsing algorithm avoids parse errors that would be redundant with validation errors. For example, <div<div> is a start tag for an element named div<div. Since there is no such element in the HTML language, the validation layer would catch the error. However, when we do not have a validation layer, the typo goes unreported.

Please do not advertise the new View Source implementation by saying that Firefox now has a validator in the View Source window.

Not All Parse Errors Are Reported!

Even though all the errors that are reported are parse errors according to the specification, not all parse errors are reported.

  • Bytes that are illegal according to the encoding of the page are not marked as errors.
  • Forbidden characters are not reported as errors.
  • Errors that relate to the end of file are not reported.
  • Tree builder errors that relate to text (as opposed to tags, comments or doctypes) are not reported.
  • Parse errors related to xmlns attributes are not reported.

XML Syntax Highlighting

The old implementation used the HTML tokenizer for highlighting XML source. So does the new implementation. While the tokenizer has support for processing instructions when it is highlighting XML source, that is the only XML-oriented additional capability. As a result, doctypes that have an internal subset are mishighlighted and entity references to custom entities are mishighlighted. This is obvious when viewing the source of Firefox chrome files. However, the mishighlighting should not be a practical problem when viewing source of typical XML files on the Web (to the extent there are XML files on the Web).

Other Known Bugs

The new implementation broke the window title for View Source windows. Also, the highlighting of the end of named character references is off by one.

Release Schedule

The code landed on trunk in time for Firefox 10. However, the landing added quite a bit of code, so it is possible that the code gets turned off after the Aurora uplift. To test the new code, using Nightly is your best bet.

Update: This feature was deferred to Firefox 11 due to regressions that were not fixed in time for Firefox 10.