Including Python dependencies with electron builder

Hello folks,

I’ve been following the link below in order to generate necessary Windows/Linux/macOS binaries. Is there any way to incorporate Python dependencies to this? Basically, some of our extensions’ backends call Python scripts which have their own dependencies. We want to make the download process as smooth as possible for the user, hence were wondering if there was any way to add like a ‘pip install requirements.txt’ to the electron-builder process?

Cheers.

Hi @kaustav-aarish,

I think this is the type of thing that is best done post-install, in the environment where the Theia app will be used, rather than being a build-time bundling of these Python dependencies, which I think could get quite complicated.

One way forward is for each extension that has such non-npm dependencies to install them the first time it’s activated. This respects the separation of concerns principle, the specific piece of code (an extension) being responsible to handle its own specific “problems”.

This can be done by launching a process that essentially run the pip install <whatever this extension needs> command on your OS and make sure it succeeded.

Alternatively you could modify the entrypoint (electron main) of your Electron app and conditionally execute a dependencies installation script that would do all such pip install commands that are required to satisfy all your extensions. Run it maybe just the first time, and then delegate to the vanilla entrypoint.
Makes sense?

@marcdumais-work,

Agreed!

Some clarification regarding your feedback:

Firstly, for the separation of concerns principle, the idea is that when an extension is triggered by the user, a process is initiated in order to install the required dependencies right? Is there any function that handles/checks the triggering of an extension?

Secondly, the “entry point modification” tactic would be to alter the main entry point such that if its the first time, it runs a script which installs necessary packages, and then after that just goes to src-gen/frontend/electron-main.js? Would this carry forward when the application is packaged as well?

Thank you so much for your help!

Have you tried to use afterPack with electron-builder?

Hi @kaustav-aarish,

First one question/clarification.

some of our extensions’ backends call Python scripts which have their own dependencies

Here, do you mean Theia extensions or vscode extensions (aka Theia Plugins)? I will assume you mean vscode extensions for the rest of the comment.

Firstly, for the separation of concerns principle, the idea is that when an extension is triggered by the user, a process is initiated in order to install the required dependencies right?

Correct - usually an action from the user will be the trigger. E.g. an extension can ask to be activated when one or more file of a certain type is opened in the IDE.

Is there any function that handles/checks the triggering of an extension?

Assuming we’re talking about vscode extensions, here is vscode documentation about activation events. You can mention which events are relevant for you extension in your package.json. e.g. vscode-clangd: https://github.com/clangd/vscode-clangd/blob/master/package.json#L25-L34

When an event that matches an extension’s activation events happens, the extension is activated, and its activate() function is called. e.g. for vscode-clangd: https://github.com/clangd/vscode-clangd/blob/master/src/extension.ts#L9

Secondly, the “entry point modification” tactic would be to alter the main entry point such that if its the first time, it runs a script which installs necessary packages, and then after that just goes to src-gen/frontend/electron-main.js? Would this carry forward when the application is packaged as well?

Yes and yes. For example, we do similar in the theia-electron example app, and this sticks in the generated packages.

1 Like

I forgot: this electron-builder.yml entry is important, else the script will not be bundled with the various packaging, I think.

1 Like

Great, I think I have a firm grasp on the solution for “entry point modification” with a lot of resources. Thanks a lot!

Here, do you mean Theia extensions or vscode extensions (aka Theia Plugins)?

Here, I’m referring to Theia extensions (ones that we built ourselves with a frontend and backend). Looking further into this, I realised that since our backend spawns a Python child process on default-server.ts, a good solution would be to trigger the dependency installation right before this to achieve the desired result? Would this accomplish the same goal as separating concerns and triggering the dependency install upon ‘activation’ of the extension?

Thanks again!

Hi @kaustav-aarish,

Here, I’m referring to Theia extensions

Thanks for confirming. Then the specifics I provided related to vscode extension activation are not applicable.

a good solution would be to trigger the dependency installation right before this to achieve the desired result?

Sounds like a good plan, or at least a starting point to experiment.

Would this accomplish the same goal as separating concerns and triggering the dependency install upon ‘activation’ of the extension?

If I understand correctly, yes. So long as the extension itself installs the dependencies it needs. But it’s mostly important if you intend to distribute your extension, to avoid requiring each user to install the dependencies themselves.

Good luck!