Differences between YstokHTML and Franz PHTML

1. Added the element-type and external-format parameters to the default parse-html method (i.e. on a pathname).
2. Changed signature of phtml-internal:
   read-sequence-func is the last optional parameter now.
3. HTML character references additions
   • Numeric and entity references are supported in (delimited) attribute values as well as in PCDATA.
   • Added pseudo-entities - non-standard character names that can be replaces by an arbitrary LHTML element.
   • The parsing code was abstracted and moved into the new parse-entity function. It converts an HTML escaped character references, i.e. numeric or named entity strings, to a
     • character,
     • string, or
     • LHTML element.
4. Function next-token
   • For state names, symbols are used instead of integers.
   • Replaced macrolet with flet.
   • Attribvaluedelim: escape char #\\ is actually skipped and next char is read.
5. Callbacks
   • Callbacks are invoked in bottom-top order, i.e. applied
     - first to leaves,
     - then to the root.
   • The callback of an element is invoked
     - for an element without an end tag - just after parsing the start tag;
     - for an element with an end tag - just after parsing the end tag (this can also be triggered during auto-closing when the start of a sibling or the end of a parent is encountered);
     - for comment - just after the whole comment is read (useful for specifying control info like TMPL_...).
   • Rewriting elements via callbacks is possible.
     A callback can return the two values:
     (1) A new element that should replace the argument received, does not matter if the second value is null.
     (2) A generalized true as an indication that the element has changed.
     :append - the new element should be revappend'ed to pending content,
     otherwise - the new element should be consed to pending content.
   • Added *callback-parent-tag* special variable. It is bound around any call of a callback.
   • Added *xhtml-mode* special variable indicating that an XHTML prologue has been encountered.
6. Pseudo-elements
   Added the following pseudo-elements similar to the !-- comment.
   • !:doctype
   • <?xml ...?>
   • <?php ...?> - in attributes, PHP code is left unparsed.
7. Marked #\? as tag-char.
8. Renamed xml-bailout to sgml-bailout, and the lexer returns the :sgml kind instead of :xml now.
9. Whitespace
   • Added function collapse-whitespace.
   • Added property tag-retain-whitespace for preformatted element symbols, e.g. <pre>.
     Spaces within the content of these elements are never collapsed.
   • Added :retain-whitespace parameter to parse-html
t - whitespace is preserved everywhere in full,
     :newline - replace #\Newline with :newline keyword and collapse or eliminate some space around it,
     nil - neither linebreaks nor other whitespace survive, except for
     - raw-mode elements: script, !DOCTYPE, etc.
     - inside the scope of tag-retain-whitespace.
10. More error reporting
    • Tracking the input line number and column similar to YHTML-Template.
    • Signaling template-syntax-error when an end tag is encountered but no matching start tag pending within the document tree.
    • Warning about absence of an explicit end tag,
    • Added template-syntax-warning conditon to issue a warning about intersection of element scopes or missing end tags. That could help to write HTML in a more strict manner.
11. Considerable code rewrite and lots of improvements
    • Removed dependency on Allegro's excl package or ACL-Compat lite: the if* macro is no longer needed.
    • More integration with HTMLGen.
    • Macro tag-no-end replaced by the inline function and implemented by means of html:*desc-hash-table*.
    • Fixed :callback-only mode, which malfunctioned in the original Franz code.