Handling Multiple/Remote Plugin Hosts

In some parts of the Theia code, we make the assumption that the plugin host process and the Theia back-end process run on the same machine or in the same container. Or more concretely, we assume that they have access to the same resources (for example files) at the same locations. The issue # Snippets don’t work when Plugin Host is not colocated with Back End is an example of where this assumption breaks in the case of Che: the file describing the Java snippets is located in the file system of a sidecar container running vscode-java in Che, not in the main Theia container.

There are some locations where we do have a notion of multiple plugin hosts in our code: for example front-end plug-ins are deployed on the theia back-end machine, but executed in the plugin host in a web worker.

There is also the notion of ServerPluginRunnerthat is used in Che to handle the case of remote plugin host processes.
So while the notion of multiple plugin hosts has seeped into Theia, it is not adhered to consistently. For example, there is a special case for “PluginHostProcess” in the back end, it isn’t just another plugin runner.

If we look at VS Code, they seem to go more to an architecture with more than one plugin hosts, as well: At least they can handle a remote plugin host, and I’ve seen code that would only make sense if there are multiple plugin hosts. For example [here](vscode/extHostAuthentication.ts at 650197b991775f5ef64a941637ba0aecf5660a0b · microsoft/vscode · GitHub): forwarding the request for sessions to the main IDE process only makes sense if there are other plugin hosts that might could fulfil that request.

With that in mind, I propose that we make the existence of multiple remote plugin hosts a design principle for Theia. Concretely, this would mostly entail two things:

First: differentiate between plugin resources and back end resources. As an example, in Che we had a problem when the C++ plugin was opening an include file from /usr/include/That file simply did not exist on the Theia back end container. Also, opening the langauge server process logs from VS Code Java would fail because they were written to a file in the Java sidecar container.

The only way Theia can know about a file that is not accessible to the back-end is if the plugin tells Theia about it either through a contribution (icon-path) or via API (like a reference to a symbol in /usr/include/stdio.h). In principle, we could treat all files as plugin-local resources, but that would lead to a weird situation: if we opened a file from the workspace, we could open it twice if we got a refererence to it from a plugin and if we opened it with File->Open. One would have an uri of file://...whereas the other would have pluginresource://.... In the cases I’m aware of, we can safely assume that the files withing a workspace root are accessible to all plugins and the back end and we could use file://...uris for those.

We would have to examine the plugin API and return the appropriate type of URI wherever URI’s are used. A file system would have to be implemented that can handle such URIs.

Note that we already have some kind of support for plugin resources for icons and such things (/hostedPlugin/<pluginId>/path uri’s).

Second: plugin API. A VS Code extension can return an API object from its start method. If a plugin is not running in the same plugin host as the API it calls, we would have to invoke that API remotely. Che has code that implements these remote calls that could be upstreamed into Theia.

Btw: there has been a question about this earlier this year: Running Hosted Plugins in a Sidecar Container