self keyword not correctly specified / highlighted

[BigfootN]

How are you using the lua-language-server?

NeoVim

Which OS are you using?

Linux

What is the issue affecting?

Other

Expected Behaviour

The self word should be highlighted as a keyword and not a variable.

Actual Behaviour

The self keyword seems to be recognized as a variable and not a keyword. The self keyword is not correctly highlighted. It seems highlighted as a variable and not as a keyword.

Reproduction steps

Use neovim with lua_ls and open a lua file and a theme with the self keyword. The self keyword is not correctly highlighted.

Additional Notes

I first noticed the problem in neovim with the everforest theme. I even opened an issue. More context can be found in the author's detailed answer.

Log File

No response

[tomlau10]

It's working correctly in vscode 😕.

And using the Developer: Inspect Editor Tokens and Scopes in vscode shows the following:

• it's semantic token type is variable.definition:lua i.e. it has a definition modifier
• however my theme doesn't have this rule, so it fallbacks to use textmate scope variable.language.self.lua
  • and my theme seems doesn't have this rule as well, so another fallback => variable.language
  • => finally a rule is matched
• I can specifically modify the color of self in my user setting json by using:
  • semantic rule: variable.definition:lua
  • or textmate scope: variable.language.self.lua

---

But all the above are vscode specific, I am not sure the case for neovim 🤔

[antoineco]

@tomlau10 no difference with Neovim as far as I can see. self is identified in Neovim as @lsp.type.variable + @lsp.typemod.variable.definition, which matches what you shared from VSCode (type: variable, modifiers: definition).

The rationale here is that self is identified as any other variable, not as a built-in variable, so it can't be highlighted differently from other variables based on LSP Semantic Tokens alone.

The workaround in Neovim is similar to what you shared: clear the highlight for lsp.type.variable when the LSP client is lua_ls, which causes a fallback to the next method: Tree-sitter if enabled, otherwise Vim's legacy syntax.

Disclaimer: I'm not very familiar with LSP Semantic Tokens so I am not even sure whether language servers have a notion of "built-in" tokens. Tree-sitter does, apparently Textmate too. The only thing that seems to come close is the defaultLibrary modifier, but I'm not sure whether that's appropriate for self because it matches on a lot of other things.

[tomlau10]

Disclaimer: I am not maintainer of LuaLS, nor have rich knowledge related to semantic tokens 🙈

---

I am not even sure whether language servers have a notion of "built-in" tokens. Tree-sitter does, apparently Textmate too.

I believe the token types are given by vscode's LSP spec 🤔 https://code.visualstudio.com/api/language-extensions/semantic-highlight-guide#standard-token-types-and-modifiers => There is no builtin type or builtin modifier

You can see that LuaLS also defined TokenType enum in that way: https://github.com/LuaLS/lua-language-server/blob/64c70846d03d5a2d7dd3b4317b8722ce232d2953/script/proto/define.lua#L142-L166

The rationale here is that self is identified as any other variable, not as a built-in variable

From the above vscode's doc, you can see that semantic token types are always used together with modifier type. So I don't think you can just do theming based on token type while ignoring its modifier? 😳 For example, you can have:

• variable.global
• variable.readonly
• variable.static
• ...

=> all the above are variables but with different attributes (modifier) And now LuaLS is giving self as variable.definition => which can be interpreted as a built-in variable (I guess)

---

Can your theme support coloring based on type + modifier 🤔 ? More specifically a rule for @lsp.type.variable + @lsp.typemod.variable.definition ? (I am not familiar with neovim)

The workaround in Neovim is similar to what you shared: clear the highlight for lsp.type.variable when the LSP client is lua_ls, which causes a fallback to the next method: Tree-sitter if enabled, otherwise Vim's legacy syntax.

If you prefer your editor's default highlight, here is another workaround:

• you can disable server's semantic coloring for variables by setting "semantic.variable": false https://luals.github.io/wiki/settings/#semanticvariable
• or even disable the whole semantic coloring feature "semantic.enable": false https://luals.github.io/wiki/settings/#semanticenable

[tomlau10]

I know that @lewis6991 is a pro user and lsp client developer in neovim, maybe he could give you some insight on this problem 👀

[antoineco]

That's what I expected, thanks @tomlau10.

Neovim applies higher priorities to types with modifiers, so you can actually be granular, but variable.definition is still quite broad despite having a modifier (a variable definition doesn't necessarily mean a built-in).

Your workaround makes sense. In fact it is similar to what I suggested @BigfootN (OP) to do.

So to summarize, if semantic tokens aren't expressive enough for the language with regards to syntax highlighting, either:

• Configure Lua LS to disable semantic tokens for that type (or entirely)
• Clear the corresponding highlight if it is set in your editor/colorscheme to fall back to the next syntax method

Issue can be closed, I guess?

[BigfootN]

Hello,

Sorry for responding so late, and thank you all for all your detailed explanation.

Indeed, the workaround @antoineco suggested to me seems to "solve" the problem.

So if I understand well, the theme should support this (from @tomlau10 ):

And now LuaLS is giving self as variable.definition => which can be interpreted as a built-in variable (I guess)

Should I close this issue ?

Thank you again for all your precious help