Theia CI

Is there currently a CI process that contains more than build, lint and test validations?

For example:

  • Security scan

  • IP scan

  • Smoke test

Are these aspects covered somewhere? Is there any pipline run in Eclipse foundation?

[original thread by amiramw]

Hi @amiramw . There is no CI being run on Eclipse Foundation hardware ATM, though it would be an option to use e.g. their Jenkins server. So far we use GitHub-related resources, like Travis CI.

We do not have a security scan as such. However it looks like GitHub does one, and the organization owners are notified when a flaw is found to impact us. I am not sure of how precise or thorough this GH scan is

There is an indirect “IP scan” being done periodically, through us registering CQs with the Eclipse Foundation, specially for recursive set of production NPM dependencies. However that scan is limited to checking for license compatibility, and does not attempt, IIUC, to verify “provenance” of our code. e.g. no check for code that might have been copied from another project, unless we register a CQ to highlight that we have done so.

If there were a decent tool to do IP scan for a project such as ours, that we could use as part of our CI on GH, I think it would be valuable to do so. I think it would need to be configurable to remember past “white listing” of results, manually confirmed to be “ok”. Then after a little while, anything reported as a potential IP problem by the tool would tend to be things that are worth investigating. If such a check is demonstrated to be precise, it could potentially be used as a trigger, for us to know that something looks fishy and that we need to register a CQ so that the Foundation can confirm if it’s ok or not.

smoke tests: we do not have ATM. It was suggested that we might run a smaller set of tests during CI, but then run a more thorough test suite, e.g. during a nightly build. The immediate need for a shorter test suite has gone down some: Recent CI stabilization reduced the chances of failures, and configuration changes made it less costly when we do have them, re-running only the suite that failed instead of the whole thing.

[amiramw]

@marc-dumais What about embedding https://www.whitesourcesoftware.com/ ? It scans open source dependencies for security and licensing issues and it is free for open source projects.

It would be interesting to give it a try, maybe on a personal fork first? Reading this : https://github.com/apps/whitesource-bolt-for-github , I understood that the IP scan is only for paying account, but it’s not 100% clear.

i.e. with “free trial”

[Deleted]

According to the FAQ they(whitesource) claim to have only one product

[Deleted]

Yes. WhiteSource offers one comprehensive solution that includes all the tools needed to ensure that you’re on top of your open source usage, including the full extent of our database with vulnerabilities from the CVE and dozens of other sources and all features (Web Advisor, unlimited number of plugins, unlimited number of users, unlimited number of policies, and more).

If someone volunteers to set this up on their theia repo fork, I would be interested in reviewing the results

[amiramw]

I will give it a try