Looking for tips to build against older Theia versions

For legal reason we are able to immediately upgrade to the latest release of Theia and need to build against a specific approved version. We also want to maintain our extensions separate from Theia clones and have development workspaces that only contain our code. To achieve that we basically follow the approach described here https://www.theia-ide.org/doc/Composing_Applications.html for building and creating workspaces plus we are building a Docker image using a script very similar to what is published here: https://github.com/theia-ide/theia-apps/blob/master/theia-docker/Dockerfile

Building against a specific version of Theia (such as 3.17) recently causes issues though when just using a package.json to select the Theia components, because the components themselves define dependencies amongst each other using “^0.3.17” instead of “0.3.17”. So even if we specify to use 3.17 the Theia components themselves will pick the newer version 3.18 and the builds break. Symptoms are errors showing incompatible version of the URI class etc. To solve that we added a long list of “resolutions” to the top-level package.json, selecting the version we want for all transitively included Theia components. This worked until 3.18 was released. Now there are new issues such as the “vscode-ripgrep: Command failed.” issue reported in another thread as well as out-of -memory errors when building the Docker image.

(1) Does anyone have experience on how to build older version and some tips for us?

(2) What is the rationale of declaring the dependencies with the “^” qualifier as mixing different versions of Theia extension does not seem to work as the URI example shows.

[original thread by Peter Haumer]

( 1) Does anyone have experience on how to build older version and some tips for us?

You should get the lock file when you installed first time against 0.3.17, if you don’t update it then yarn should not pull never versions. If you don’t have the lock file yet, then yes resolutions is the way to go.

(2) What is the rationale of declaring the dependencies with the “^” qualifier as mixing different versions of Theia extension does not seem to work as the URI example shows.

I cannot remember now. We had exact versions but when it caused some issues. It could be related to the dynamic ability to install extensions, but since it is not used anymore, maybe we can reconsider and pin to exact.

But only in the Theia repo, for external extensions it would require their update with each version.

[Peter Haumer]

Thanks a lot @anton-kosyakov. I will create an Issue requesting exact versions.

Do you or anyone else have any experience with where to start analysing a sudden occurence of this oom error during the docker build? One thing I observed is that node:8-alpine was now using a newer version, but switching back to 8.12 also did not make a difference. What would be the place to configure memory for node in Theia docker build? (Docker itself is configured with 5GB)

What else could cause this. With all those versions changing automatically build seems to becoming a game of chance :slight_smile:

     [exec] [4/4] Building fresh packages...
     [exec] Done in 95.38s.
     [exec] yarn run v1.9.4
     [exec] $ /home/wazi/node_modules/.bin/theia build
     [exec]
     [exec] <--- Last few GCs --->
     [exec]
     [exec] [1574:0x564e8c087000]    72807 ms: Mark-sweep 1422.9 (1517.7) -> 1422.8 (1517.7) MB, 685.1 / 0.0 ms  allocation failure GC in old space requested
     [exec] [1574:0x564e8c087000]    73668 ms: Mark-sweep 1422.8 (1517.7) -> 1422.7 (1486.7) MB, 860.4 / 0.0 ms  last resort GC in old space requested
     [exec] [1574:0x564e8c087000]    74472 ms: Mark-sweep 1422.7 (1486.7) -> 1422.7 (1486.2) MB, 803.6 / 0.0 ms  last resort GC in old space requested
     [exec]
     [exec]
     [exec] <--- JS stacktrace --->
     [exec]
     [exec] ==== JS stack trace =========================================
     [exec]
     [exec] Security context: 0x3193f73a5879 <JSObject>
     [exec]     1: _parseMappings(aka SourceMapConsumer_parseMappings) [/home/wazi/node_modules/webpack-sources/node_modules/source-map/lib/source-map-consumer.js:~468] [pc=0x505f1f4cd41](this=0x128d522254e1 <BasicSourceMapConsumer map = 0x126ad6071599>,aStr=0x9145ce40301 <Very long string[1806]>,aSourceRoot=0x6cf92e02201 <null>)
     [exec]     2: get [/home/wazi/node_modules/webpack-sources/node_modules/source-map/li...
     [exec]
     [exec]
     [exec] FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
     [exec]
     [exec] Error: webpack exited with an unexpected signal: SIGABRT.
     [exec]     at ChildProcess.<anonymous> (/home/wazi/node_modules/@theia/application-manager/lib/application-process.js:59:28)
     [exec]     at emitTwo (events.js:126:13)
     [exec]     at ChildProcess.emit (events.js:214:7)
     [exec]     at maybeClose (internal/child_process.js:915:16)
     [exec]     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
     [exec] error Command failed with exit code 1.
     [exec] info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
     [exec] The command '/bin/sh -c yarn --pure-lockfile &&     yarn theia build && yarn --production &&     yarn autoclean --init &&     echo *.ts >> .yarnclean &&     echo *.tsx >> .yarnclean &&     echo *.ts.map >> .yarnclean &&     echo *.spec.* >> .yarnclean &&     yarn autoclean --force &&     rm -rf ./node_modules/electron &&     yarncache clean &&     rm -rf @ibm' returned a non-zero code: 1

Have you tried to configure node with more memory?

You can try to remove extension one by one and run build to find one which triggers it, and then investigate deeper.

[Peter Haumer]

Thanks. Do you have a recommendation how and where to configure node memory? Is that something that can be done with yarn parameters or which script would I need to change how?

Hi, there were env variable, let me look it up for you

export NODE_OPTIONS=--max_old_space_size=4096

[Peter Haumer]

Thanks a lot, that eliminated the oom error. I also realized that a key difference in your Docker build was that you were building there without a yarn.lock file. After I changed the scipt to copy ours into the image for building it worked.

glad that you resolved your issue :slight_smile:

Anton, I see similar out of memory traces in some of our docker docker images. Thanks for sharing.

would be also interesting to find out which extensions cause it, maybe they can be improved