Migrate lexical-code from PrismJS to Shiki

[ivailop7]

PrismJS appears to be no longer maintained and Shiki seems to be the closest maintained alternative for syntax highlighting. A number of people have suggested the migration and I agree it makes sense, primarily because PrismJS is not ESM and creates issues when used with certain server-side rendering frameworks.

[etrepum]

I have looked into this a little bit, the downsides are:

• Shiki is larger since it brings in a wasm build of the oniguruma regex engine (~500kb gzip I think) required to support the TextMate grammars that all of the syntax highlighting is defined with https://shiki.style/guide/bundles
• It's designed more for ahead-of-time highlighting rather than as-you-type highlighting
• It uses inline style rather than semantic classes

Probably not a deal-breaker for everyone but I assume that there are many people that would prefer the dead-end prismjs solution at <100kb

[ivailop7]

Not tied to the framework, but it appears everyone I've spoken to recently is switching to this one, short of Monaco and CodeMirror embeds. Size-wise - yes, 5x increase is significant. If there are other reasonable alternatives, I'm all ears.

[etrepum]

I didn't find anything better, which was disappointing. Maybe the ideal would be to properly divorce highlighting from the code node model so that either solution (or something bespoke, e.g. a custom treesitter based highlighter) could be used. For the most part it really is just a display concern, shouldn't need such tight coupling.

[etrepum]

Looks like https://github.com/wooorm/starry-night is a similar project to shiki but AST and class based

[etrepum]

Looks like the size concerns with Shiki are being addressed https://github.com/shikijs/shiki/pull/761

[rockwotj]

Also - could the list of languages be made pluggable? If I want to support a different subset of prism langauges than the existing lexical-code package, I have to fork or patch the list of languages. Would be awesome if I could customize that without the fork/patch.

[etrepum]

Given that everything in prism is global you should be able to add languages by importing them before you import lexical code. Removing them is a bit trickier but you could configure your bundler to make some of them return an empty module or to eliminate those imports altogether.

[vadimkantorov]

Also, I've had to import Prism like so:

import Prism from "prismjs"; if (typeof globalThis.Prism === 'undefined') { globalThis.Prism = Prism;}
import {CodeNode} from '@lexical/code';

Otherwise, I was getting an exception at runtime of global variable Prism not available:

• https://github.com/remix-run/remix/discussions/8182

(I was needing @lexical/code and Prism to have some basic triple-backtick-block rendering in @lexical/markdown, unfortunately currently @lexical/markdown and @lexical/code do not work without installed and global Prism)

[kyoshino]

Lexical v0.34.0 comes with experimental Shiki support.

[vadimkantorov]

to have some basic triple-backtick-block rendering in @lexical/markdown

Hopefully, triple-backticks in @lexical/markdown can now be rendered without highlighting / depending on @lexical/code too (especially if highlighting adds quite a large payload)

[etrepum]

For compatibility reasons prismjs is included in the bundle whether you use it or not, unless you configure your bundler to eliminate it accordingly. It has always been possible to not enable highlighting.