Customizing TextMate scopes for semantic coloring

#1

Hi there! I’m working on semantic coloring support for C++, and have a couple of questions.

Semantic coloring is a proposed LSP feature with client-side support in Theia. It works by having the server send token ranges along with “TextMate scopes” which identify the coloring to be used for each token.

As far as I can tell, the colors are defined in theme files such as packages/monaco/data/monaco-themes/vscode/dark_vs.json. For example, here that theme file associates the scope “meta.preprocessor” with a particular foreground color.

Now, I have two questions:

  • If I would like the C++ plugin to add support for some C+±specific scopes, and associated default colors, how can I do that?

  • If then I as a user of the C++ plugin would like to customize the color for a particular scope to something other than the default, how can I do that?

Thanks!

cc @kittaakos

[original thread by Nathan Ridge]

#2

  1. You can provide a list of scopes per token. They are matched against the active color scheme. See https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide#textmate-grammars

  2. To add new colors and scopes you need to provide and register a color theme. See https://code.visualstudio.com/api/extension-guides/color-theme#syntax-colors

#3

Thanks, Sven.
@nridge, please note the following limitations:

#4

[Nathan Ridge]

@kittaakos, thanks for letting me know about that.

Do I understand correctly that these issues mean that the semantic highlighting feature does not currently work (or does not work reliably) until the plan described in this comment is implemented?

#5

[Nathan Ridge]

I don’t fully understand the relationship between Theia and VSCode, and so I don’t know to what extent problems and solutions carry over from one to the other, but I’ll note that vscode-cquery has a fully functional semantic highlighting feature (using a custom LSP protocol extension, not your proposed one). This is the API it uses to apply the highlightings. Does that help us at all?

#6

[Nathan Ridge]

@svenefftinge, thanks for the pointer. However, I’m a bit confused as this talks about using editor.tokenColorCustomizations. That is indeed how I do it in VSCode, but Theia doesn’t seem to recognize it.

#7

However, I’m a bit confused as this talks about using editor.tokenColorCustomizations. That is indeed how I do it in VSCode, but Theia doesn’t seem to recognize it.

@nridge Theia is capable of running VS Code extensions. Could you try to run your VS Code extensions? If somethign does not work, we should file a bug and look into it. On how to test: https://github.com/theia-ide/theia/wiki/Testing-VS-Code-extensions

#8

Looking at source code of vscode-cquery, it seems that all used vscode APIs are alredy implemented, except 2 commands workbench.action.openWorkspaceSettings and workbench.action.reloadWindow.

#9

[Nathan Ridge]

@anton-kosyakov, I’m not looking to port vscode-cquery to Theia (it’s a plugin for interfacing with the “cquery” language server whose development has largely been abandoned in favour of clangd). I was just pointing out that it successfully implements semantic coloring. I see that it uses the TextEditor.setDecorations() API. Do I understand correctly, that we can now use this API in Theia? @kittaakos, would our ability to use this API (instead of deltaDecorations(), I guess?) resolve the issue you mentioned?

#10

You can run VS Code extensions using this API in Theia, for instance power-mode ext uses it: https://github.com/theia-ide/theia/pull/4827

#11

[Nathan Ridge]

Ok, I see. The place where we’d want to use it, though, is the shared (cross-language) client implementation of an LSP feature. Is that allowed to be in a VSCode plugin?

#12

[Nathan Ridge]

Following up on another subtread: @svenefftinge, I’ve had a look at https://code.visualstudio.com/api/extension-guides/color-theme#syntax-colors, and I’m still a bit confused about how it’s applicable to the problem of a Theia plugin extending a color theme. Could you perhaps point me to an existing Theia plugin which extends a color theme?

#13

is the shared (cross-language) client implementation of an LSP feature. Is that allowed to be in a VSCode plugin?

#14

This API is not part of LSP, so if you want to build a part which is reusable for different editors, not only for IDEs supporting VS Code extensions then it is not a way to go :frowning:

#15

when you have to go with https://github.com/Microsoft/vscode-languageserver-node/pull/367 and its limitations in Theia as @kittaakos mentioned

#16

[Nathan Ridge]

I’m not looking to “build a part which is reusable for different editors”, I’m looking to implement a particular LSP message (textDocument/semanticHighlighting) in a particular editor (Theia), or rather fix an issue in its existing implementation.