update keyword and operator definitions in highlights.scm

[fdncred]

This PR is an attempt to update the keywords, operators and commands. I'm hoping that things like str replace are highlighted as a command instead of being highlighted as a command and an argument.

I'm not sure how to test this. Any guidance would be helpful.

Maybe commands with spaces aren't allowed?

[blindFS]

I think it's not possible without changing the grammar. And even if we manage to make builtin internals correct, there're still user defined composite commands... I'm afraid there's no perfect solution for this in tree-sitter.

[fdncred]

I got the CI to pass at least. I guess I need to point my nvim config at this branch to get it to work/use it?

[blindFS]

I got the CI to pass at least. I guess I need to point my nvim config at this branch to get it to work/use it?

Actually you can just edit your highlights.scm file in the ftplugin to test the color, but I think nothing will change, since the parsed tree is the same.

[fdncred]

no clue what ftplugin is. this is my config https://github.com/fdncred/nvim-dotfiles/blob/de092ea37c325545ffca85d702c8e323e28ad979/lua/kickstart/plugins/treesitter.lua#L33

[blindFS]

I mean the local path that your plugin manager installs tree-sitter-nu into, in my case, it's ~/.local/share/nvim/lazy/tree-sitter-nu/queries/nu, I've tested the command head, no diff:

[fdncred]

ya, right. it's pointing to my local where the file is edited.

darn. so, the grammar has to be changed to make the highlighting work right? i guess that means that the grammar doesn't recognize commands with spaces.

I guess i don't understand this part of the highlights.scm if it does nothing. why is it even in the file?

[blindFS]

@fdncred it does the job of correct highlighting the "head" if it is in the collected strings, however the replace is parsed as argument already, it's out of head, so won't be affected.

[blindFS]

Sorry, you're right, (cmd_identifier) already get it highlighted, the second rule is totally redundant.

[fdncred]

would be nice if the grammar for nushell would be able to highlight commands in nushell. LOL

Seems like the parser would need to check if the text after a command was part of the command or not and if not it's an argument, otherwise it's all the command. Maybe that's easier to say than to do?

[blindFS]

@fdncred The only way that I can imagine is hardcoded choices in command_rule:

function _command_rule(parenthesized) {
  return (/** @type {any} */ $) => {
    const sep = parenthesized ? $._separator : $._space;
    return prec.right(
      seq(
        choice(
          field("head", seq(optional(PUNC().caret), $.cmd_identifier)),
          field("head", seq(PUNC().caret, $._stringish)),
          field("head", choice(...INTERNALS())), // <-- collection of all composite commands
        ),
        repeat(seq(sep, optional($._cmd_arg))),
      ),
    );
  };
}

But still, it can't deal with user-defined ones (probably no solution).

[fdncred]

I'm not as concerned about custom commands, but internal commands and plugins should at least be identified as such.

[mrdgo]

I see two options:

1. full enumeration of all the commands with spaces within the grammar
2. Retrieve the commands with spaces from upstream
   Both have pros and cons, I'd open a discussion with the treesitter people, they might know, if 2. is even possible.

Edit: I have a slight aversion against manually maintaining all these commands in this repo

[fdncred]

Edit: I have a slight aversion against manually maintaining all these commands in this repo

I understand that but I'd also "keep it simple". Having the commands and plugin command hard coded somehow seems easy to maintain because that section could just be replaced as needed. This is what the vscode extension does here in the match line. A tool was written to generate this regex monstrosity.

[mrdgo]

Edit: I have a slight aversion against manually maintaining all these commands in this repo

I understand that but I'd also "keep it simple". Having the commands and plugin command hard coded somehow seems easy to maintain because that section could just be replaced as needed. This is what the vscode extension does here in the match line. A tool was written to generate this regex monstrosity.

We could also use the regex, then we only sync regexes and don't have to maintain different representations of the same information. What do you think?

[fdncred]

I'm up for trying it. If we don't like it, we can always revert it.

[blindFS]

@fdncred naive implementation FYI.

Update: extra regex suffix for situations like weatherdark.nu where http get_api has overlap with internal http get, causing a parsing error each time http get_api is called without the fix.

[fdncred]

With vscode, we can make regexes with ^/$ and \b word boundaries. Would be nice to be able to combine INTERNAL with word boundaries so it knows where words end. Also, can you put up a PR with your implementation. I think I could test it easier by checking out your PR.

[blindFS]

With vscode, we can make regexes with ^/$ and \b word boundaries. Would be nice to be able to combine INTERNAL with word boundaries so it knows where words end. Also, can you put up a PR with your implementation. I think I could test it easier by checking out your PR.

Sad story, \b is not supported in tree-sitter grammar

Error processing rule command_token1

Caused by:
    Regex error: Assertions are not supported