feat: fallback to regex highlighting when treesitter is not available #462
Conversation
Issue ===== Documentation for completions can include markdown fenced code blocks that should be highlighted with the appropriate syntax. When a treesitter parser is not available for the language used in the codeblock, the code inside is not highlighted correctly. This can be an issue when using https://github.com/mikavilpas/blink-ripgrep.nvim which displays ripgrep (rg) matches for all the files in the project. Many big projects include a large variety of programming languages, and the user may not have all the treesitter parsers installed for all the languages. Solution ======== Right now, neovim comes bundled with a regex-based syntax highlighter that supports 743 languages. We can use this syntax highlighter as a fallback when a treesitter parser is not available for the language. The solution is not perfect but an 80% solution. The entire documentation text is highlighted in the syntax of the first codeblock. This means the code will be displayed correctly, but any markdown syntax in the documentation will look weird. Related issue: Saghen#336
Right now this also needs support from blink. See here: Saghen/blink.cmp#462
|
Hmm I have been thinking that my use case might be quite special and it probably isn't the best at addressing regex based highlighting in blink in general. Maybe there's another way to solve this. For example, to get colored highlights, I could provide a function in blink-ripgrep that blink could call when the documentation needs to be highlighted. That way I could also add other decorations that are not supported by regex (or treesitter). For example, I would like to highlight the matched word with a different background color. Let me know what you would prefer! |
|
I think a function makes sense. It could look something like |
|
Sounds good 👍🏻 I'll follow that PR, it looks like it enables a huge number of things that I like. |
|
This was implemented in #650 using a different approach. |
See here for the blink side implementation: Saghen/blink.cmp#462
When a result is found for a file whose filetype does not have a treesitter parser installed, fall back to regex based highlighting that is bundled in Neovim. This allows highlighting many more file types out of the box. See here for the blink side implementation and a mini demo: Saghen/blink.cmp#462
Issue
Documentation for completions can include markdown fenced code blocks
that should be highlighted with the appropriate syntax. When a
treesitter parser is not available for the language used in the
codeblock, the code inside is not highlighted correctly.
Incorrect highlighting example:
This can be an issue when using
https://github.com/mikavilpas/blink-ripgrep.nvim which displays ripgrep
(rg) matches for all the files in the project. Many big projects include
a large variety of programming languages, and the user may not have all
the treesitter parsers installed for all the languages.
Solution
Right now, neovim comes bundled with a regex-based syntax highlighter
that supports 743 languages. We can use this syntax highlighter as a
fallback when a treesitter parser is not available for the language.
Improved highlighting example:
The solution is not perfect but an 80% solution. The entire
documentation text is highlighted in the syntax of the first codeblock.
This means the code will be displayed correctly, but any markdown syntax
in the documentation will look weird.
The idea is to provide a "better than nothing" experience out of the box, but if the user wants better highlighting, they should install the parser(s) for the languages they are interested in 🙂
Related issue: #336