Codec related error when trying to build Theia from source

Hello, could anyone please shed some light? Thanks!

$ node-gyp install
gyp info it worked if it ends with ok
gyp info using node-gyp@3.8.0
gyp info using node@10.16.0 | linux | x64
gyp http GET https://nodejs.org/download/release/v10.16.0/node-v10.16.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v10.16.0/node-v10.16.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v10.16.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v10.16.0/SHASUMS256.txt
10.16.0
gyp info ok
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
warning monaco-languageclient@0.9.0: The engine "vscode" appears to be invalid.
warning vscode-base-languageclient@4.4.0: The engine "vscode" appears to be invalid.
info fsevents@1.2.4: The platform "linux" is incompatible with this module.
info "fsevents@1.2.4" is an optional dependency and failed compatibility check. Excluding it from installation.
[4/5] Linking dependencies...
warning " > istanbul-instrumenter-loader@3.0.1" has unmet peer dependency "webpack@^2.0.0 || ^3.0.0 || ^4.0.0".
warning " > tslint-language-service@0.9.9" has incorrect peer dependency "typescript@>= 2.3.1 < 3".
warning "workspace-aggregator-dd0f4247-ac29-4d20-878d-110490e7b17a > @theia/application-manager > font-awesome-webpack@0.0.5-beta.2" has unmet peer dependency "font-awesome@>=4.3.0".
warning "workspace-aggregator-dd0f4247-ac29-4d20-878d-110490e7b17a > @theia/debug > requestretry@3.1.0" has unmet peer dependency "request@2.<em>.</em>".
warning "workspace-aggregator-dd0f4247-ac29-4d20-878d-110490e7b17a > @theia/application-manager > font-awesome-webpack > less-loader@2.2.3" has incorrect peer dependency "less@^2.3.1".
[5/5] Building fresh packages...
[10/14] ⢀ find-git-repositories
[8/14] ⠠ oniguruma
[-/14] ⠠ waiting...
[11/14] ⠠ @theia/electron
error /mnt/c/Projects/theia-source/node_modules/@theia/electron: Command failed.
Exit code: 127
Command: node scripts/post-install.js
Arguments:
Directory: /mnt/c/Projects/theia-source/node_modules/@theia/electron
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@3.8.0
gyp info using node@10.16.0 | linux | x64
make: Entering directory '/mnt/c/Projects/theia-source/dev-packages/electron/native/build'
CC(target) Release/obj.target/ffmpeg/src/ffmpeg.o
gyp info spawn /usr/bin/python2
gyp info spawn args [ '/mnt/c/Projects/theia-source/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/mnt/c/Projects/theia-source/dev-packages/electron/native/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/mnt/c/Projects/theia-source/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/home/dossantosr/.node-gyp/10.16.0/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/home/dossantosr/.node-gyp/10.16.0',
gyp info spawn args '-Dnode_gyp_dir=/mnt/c/Projects/theia-source/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/home/dossantosr/.node-gyp/10.16.0/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/mnt/c/Projects/theia-source/dev-packages/electron/native',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.' ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
CC(target) Release/obj.target/ffmpeg/src/linux-ffmpeg.o
SOLINK_MODULE(target) Release/obj.target/ffmpeg.node
COPY Release/ffmpeg.node
make: Leaving directory '/mnt/c/Projects/theia-source/dev-packages/electron/native/build'
gyp info ok
Found cached ffmpeg library.
Hashes are equal, not replacing the ffmpeg library.
Error: 2 bad / 18 found

> h264 detected (H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10)
aac detected (AAC (Advanced Audio Coding))
at main (/mnt/c/Projects/theia-source/dev-packages/electron/electron-codecs-test.js:64:15)
at Object.<anonymous> (/mnt/c/Projects/theia-source/dev-packages/electron/electron-codecs-test.js:70:1)
at Module._compile (internal/modules/cjs/loader.js:776:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
at startup (internal/bootstrap/node.js:283:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)
Error: "electron-codecs-test.js" exited with 127
at ChildProcess.subprocess.once (/mnt/c/Projects/theia-source/dev-packages/electron/scripts/post-install.js:36:40)```

<em>[<a href="https://spectrum.chat/theia/dev/codec-related-error-when-trying-to-build-theia-from-source~0770f11a-5cdc-4c68-bd3f-39c1d2b08de5">original thread</a> by Raphael Dos Santos]</em>

Hi @santosr . A little while ago we were asked by the Eclipse Foundation to add a mechanism to check that “libffmpeg” we use in Theia apps does not contain proprietary codecs that may have some content under patent. As part of this we have a mechanism that replaces “libffmpeg” pulled for Electron with a version that does not contain such codecs. Then we have a script that tests “libffmpeg” and fails if the proprietary codecs are found. It looks like this test is failing for you.

If you do not care about codecs, we have a way to disable the library replacement / check with an environment variable: THEIA_ELECTRON_SKIP_REPLACE_FFMPEG=1

see here for a bit more details: https://github.com/theia-ide/theia/blob/master/dev-packages/electron/README.md

[Raphael Dos Santos]

Hi @marc-dumais , I have set the env var above as you suggested, but I still getting the exact same error when trying to build.

Can you tell us if you have something under /mnt/c/Projects/theia-source/dev-packages/electron/download, if yes then report here and delete it. Basically we try to download a “clean” ffmpeg library, but something seems to be wrong from time to time :confused:

[Raphael Dos Santos]

Hi @paul-marechal , a file libffmpeg.so shows up there when building. I can progress a bit further when removing it before running the build, but then I get stuck at download-jdt-ls.js

[5/5] Building fresh packages…
[-/14] ⠐ waiting…
[12/14] ⠐ @theia/debug-nodejs
[13/14] ⠐ @theia/java
[-/14] ⠐ waiting…
error /mnt/c/Projects/theia-source/node_modules/@theia/java: Command failed.
Exit code: 1
Command: node ./scripts/download-jdt-ls.js
Arguments:

Your issue with the libffmpeg.so really confuses me, as it seems like a file gets downloaded, but from time to time it is not a “clean” library, no idea why. GitHub serving the wrong file? I really don’t understand.
But once you have this clean lib downloaded, you should be clear from meddling with it for some time.

Regarding jdt-ls I am not familiar at all with this… cc @florentbenoit ?

Ah regarding the ffmpeg issue, turns out there is a little quirk when hashing in NodeJS, will fix that very soon, my bad :frowning:

[Raphael Dos Santos]

Thanks for the feedback @paul-marechal .

I’m still stuck with download-jdt-ls.js, but a co-worker on MacOS managed to compile Theia from source without hassle. Seems like it’s a Linux issue.

@paul-marechal has found a bug in the libffmpeg validation script, that may explain the issue above, and will work on a fix ASAP. The problem with jdt.ls would be distinct, but sounds more like a temporary issue (network or hosting) than a bug.

BTW, until the fix reaches you, the work-around whenever this happens is to delete file /mnt/c/Projects/theia-source/dev-packages/electron/download/libffmpeg.so