Whitespace is discarded in token stream
Lossless Parsing with Whitespace Preservation
It looks like the lexer discards the whitespace. This should not be discarded and rather sent to other hidden channel. This results in loss of information. Downsides:
- Cannot reconstruct the exact original source from the parse tree
- Build accurate code formatters
- Develop linters that understand whitespace context
https://github.com/jknack/handlebars.java/blob/master/handlebars/src/main/antlr4/com/github/jknack/handlebars/internal/HbsLexer.g4#L376
WS
: [ \t\r\n] -> skip
;
I see this might be inspired from mustache: https://github.com/jknack/handlebars.java/blob/master/handlebars/src/main/java/com/github/jknack/handlebars/internal/MustacheSpec.java
However this can be discarded in TemplateBuilder.java
@jknack Could you please review the enhancement for re-constructing source content without loss? pr: https://github.com/jknack/handlebars.java/pull/1169
Thank you