Page MenuHomePhabricator

wikidata-query-gui build tooling uses outdated packages (via grunt)
Open, Needs TriagePublic5 Estimated Story Points

Description

The wikidata-query-gui still uses grunt to run its tests, build itself, and deploy itself.
Some packages in this pipeline are completely unmaintained like grunt-usemin.

grunt-usemin has a old version of lodash in its dependency chain which causes security alerts in npm audit due to Prototype Pollution.
While this is not a security issue for us, it is noise in the npm audit reports and the weekly dependabot security emails.

Acceptance Criteria:

  • packages used as part of the build tooling are up to date per npm audit

Event Timeline

Addshore subscribed.

In ticket polishing it was said that we might want to stop using grunt in all of out codebases.

Addshore renamed this task from Replace wikidata-query-gui build tooling to wikidata-query-gui build tooling uses outdated packages (via grunt).Feb 25 2021, 10:23 AM
Addshore updated the task description. (Show Details)

Task Inspection notes:

  1. Try to find a package similar to grunt-useman and replace it due to the security issues.
  2. If such package does not exist, stop using grunt for the build step and do it with npm/webpack etc.

Findings so far:

  • The build task consist of several steps: less, copy, several usemin subtasks and merge-i18n.
  • The grunt-usemin package is some kind of umbrella task that covers several build steps, including concat, cssmin, uglify, filerev and htmlmin.
  • It is not a good option to replace only grunt-usemin with some other grunt task(s), leaving grunt itself and its remaining tasks untouched, because apparently usemin was a widely used de-facto standard approach, back in the days. In order to replace it, we would have to go and find a different not-so-widely used grunt task which will probably also be outdated soon, as people move away from grunt.
  • A possible alternative to using grunt is running the entire build process by using npm scripts directly:
    • grunt-contrib-clean, grunt-contrib-less and grunt-contrib-copy can easily be replaced by npm modules del-cli, less and cpy-cli
    • For grunt-usemin there is also a non-grunt CLI version called usemin-cli that is not outdated, but unfortunately it does not cover the filerev step.
    • I was not able to find a good replacement for the grunt-filerev task, which is a grunt-usemin step and also deprecated already. The best candidate is node-file-rev, which is a CLI tool, but it only covers revisioning and renaming files, not replacing their occurrences (as grunt-filerev does in combination with usemin).
    • The build step merge-i18n is also a grunt task, which was built by wikimedia. It is hosted on gerrit and mirrored to GitHub, from where it is loaded by wikidata-query-gui's package.json. It can potentially be left untouched and re-used in combination with an npm module called grunty, which allows running grunt tasks directly via npm scripts, without setting up grunt and configuring a big Gruntfile.js.
  • If it turns out that the entire build process can not be covered by npmscripts directly, then webpack is probably a good alternative. It can possibly be combined with grunty to run the merge-i18n step. Apparently even a usemin plugin for webpack exists, not sure if that is helpful.

Three options left:

  1. do it via npm scripts
  2. convert the build process to webpack
  3. don't do anything, because the lodash security vulnerability only affects our own build process and does not end up on production

Camp currently favours option 3. @Addshore? @Michael?

Option 3 sounds fine to me.
I'm sure one day this will be converted to webpacke / whatever build system we are using then, but perhaps today is not that day

Removing this from the camp board since option 3 was agreed on 2 weeks ago and nobody has made an objection since then

Silvan_WMDE subscribed.

forgot to unassign myself when moving this task back to ToDo :-)