Page MenuHomePhabricator

Investigate replacing vue-cli with vite and webpack with rollup for Toolhub
Open, LowPublic

Description

Security Review Summary - T273020 - 2021-08-09
Last commit reviewed: d9e475d1ff13

[...snip...]

Build/Test steps
Currently, the code uses vue-cli-services to build its dist artifacts, which by default uses certain webpack dependencies. Given the complexity and known code quality issues of webpack, this will be categorized as a Medium Risk. I'm not certain of the end destination of Toolhub, but if it is to be hosted within "Wikimedia production", then the Security-Team (@Reedy, @Mstyles or myself) should likely be apprised of any webpack-related builds that go through gerrit, e.g. https://gerrit.wikimedia.org/r/641052 and https://gerrit.wikimedia.org/r/708629. Otherwise, the current recommended mitigation is to use vite/rollup as an alternative to webpack (e.g. T276366).

The patches for T276366: Wikidata Query Builder: replace vue-cli with vite and webpack with rollup also included a switch from node v10 to node v12. If that is a strict dependency this would be blocked on T284352: Upgrade Toolhub ui container from nodejs10 to nodejs12 (or newer).

Event Timeline

bd808 triaged this task as Low priority.Aug 10 2021, 5:42 PM
bd808 created this task.
bd808 changed the task status from Open to Stalled.Aug 12 2021, 12:15 AM

Blocked by T284352: Upgrade Toolhub ui container from nodejs10 to nodejs12 (or newer). Node v10 is dead to the node developer community having fallen out of upstream support completely on 2021-04-30.

bd808 changed the task status from Stalled to In Progress.Feb 28 2022, 4:36 PM
bd808 claimed this task.
bd808 changed the status of subtask T284352: Upgrade Toolhub ui container from nodejs10 to nodejs12 (or newer) from Open to In Progress.
bd808 moved this task from Backlog to In Progress on the Toolhub board.

After spending about 6 hours working on this idea, I am going to unwind my local changes and return this task to the backlog. I doubt that this migration is impossible, but it has some challenges that are not trivially solved yet.

I started off following the general advice from https://medium.com/nerd-for-tech/from-vue-cli-to-vitejs-648d2f5e031d. I also found https://github.com/MrBin99/django-vite to replace https://github.com/django-webpack/django-webpack-loader on the Django side.

  • Imports of .vue files need to explicitly use the file extension or be worked around with a plugin. Using the explicit extension is probably the best fix. I have seen claims that extension-less Vue imports will be dropped in a future vue-cli as well.
  • Rollup does not natively support require() for loading modules. The @originjs/vite-plugin-commonjs and @originjs/vite-plugin-require-context plugins can be used to work around this where code cannot be easily rewritten to use import. I am sill having issues with the usage of require.context() in loadLocaleMessages() from vue/src/plugins/i18n/index.js related to the file paths generated by /@originjs/vite-plugin-require-context//.
  • I have not been able to find a replacement for @vue/cli-plugin-unit-mocha or more specifically the correct configuration to build the codebase before invoking mocha to test it.
vite.config.js
const fs = require( 'fs' );
const path = require( 'path' );

import { defineConfig } from 'vite';
import { createVuePlugin } from 'vite-plugin-vue2';
import envCompatible from 'vite-plugin-env-compatible';
import { viteCommonjs } from '@originjs/vite-plugin-commonjs';
import ViteRequireContext from '@originjs/vite-plugin-require-context';

const PORT = process.env.PORT || 8001;

// T280069: Export the current git hash as an env var
process.env.VUE_APP_VERSION = require( './package.json' ).version;
process.env.VUE_APP_GIT_HASH = ( function () {
        try {
                const HEAD = fs.readFileSync(
                        path.resolve( __dirname, '.git/HEAD' ), 'utf8'
                );
                if ( HEAD.indexOf( ':' ) !== -1 ) {
                        // HEAD is a branch pointer, not a direct hash
                        const branch = HEAD.split( ': ' )[ 1 ].trim();
                        return fs.readFileSync(
                                path.resolve( __dirname, '.git/' + branch ), 'utf8'
                        ).substr( 0, 7 );
                } else {
                        return HEAD.substr( 0, 7 );
                }
        } catch ( e ) {
                // eslint-disable-next-line no-console
                con




export default defineConfig( {
        plugins: [
                createVuePlugin(),
                envCompatible(),
                viteCommonjs(),
                ViteRequireContext( {
                        projectBasePath: path.resolve( __dirname, 'vue' )
                } )
        ],
        root: path.resolve( __dirname, 'vue/src' ),
        base: '/static/',
        resolve: {
                alias: [
                        {
                                find: '@',
                                replacement: path.resolve( __dirname, 'vue/src' )
                        }
                ]
        },
        build: {
                outDir: path.resolve( __dirname, 'vue/dist' ),
                assetsDir: '',
                manifest: true,
                emptyOutDir: true,
                target: 'es2015',
                rollupOptions: {
                        input: {
                                main: path.resolve( __dirname, 'vue/src/main.js' )
                        },
                        output: {
                                manualChunks( id ) {
                                        if ( id.includes( 'node_modules' ) ) {
                                                return 'chunk-vendors';
                                        }
                                }
                        }
                }
        },
        css: {
                postcss: {
                        plugins: [
                                { // https://github.com/vitejs/vite/issues/6333
                                        postcssPlugin: 'internal:charset-removal',
                                        AtRule: {
                                                charset: ( atRule ) => {
                                                        if ( atRule.name === 'charset' ) {
                                                                atRule.remove();
                                                        }
                                                }
                                        }
                                }
                        ]
                }
        },
        server: {
                host: '0.0.0.0',
                port: PORT,
                strictPort: true,
                open: false,
                cors: { origin: '*' },
                watch: {
                        usePolling: true,
                        disableGlobbing: false
                }
        }
} );
git diff package.json
diff --git i/package.json w/package.json
index d3d2b8a..ad03394 100644
--- i/package.json
+++ w/package.json
@@ -3,19 +3,19 @@
     "version": "1.0.0",
     "private": true,
     "scripts": {
-        "build:vue": "vue-cli-service build",
+        "build:vue": "vite build",
         "format": "eslint --fix .",
         "lint": "npm run lint:eslint && npm run lint:vue && npm run lint:stylelint && npm run lint:banana && npm run lint:css-rtl",
         "lint:banana": "banana-checker vue/src/assets/locales/i18n/",
         "lint:css-rtl": "bash -c \"if grep -rnHIE --color '\\b[mp][lr]-(n?[0-9][0-6]?|auto)\\b' vue/src; then echo 'ERROR: Use S and E helper classes, not L and R. (T269056)'; echo; exit 1; fi\"",
         "lint:eslint": "eslint .",
         "lint:stylelint": "stylelint -f verbose '**/*.{css,scss,sass,vue}'",
-        "lint:vue": "vue-cli-service lint",
+        "lint:vue": "eslint",
         "schemas:generate": "jsonschema-tools materialize-all",
-        "serve:vue": "vue-cli-service serve",
+        "serve:vue": "vite",
         "unit": "npm run unit:jsonschema && npm run unit:vue",
         "unit:jsonschema": "mocha tests/jsonschema",
-        "unit:vue": "NODE_ENV=test nyc vue-cli-service test:unit --colors 'vue/src/**/*.spec.js'",
+        "unit:vue": "NODE_ENV=test nyc mocha --colors 'vue/src/**/*.spec.js'",
         "test": "npm run lint && npm run unit"
     },
     "dependencies": {
@@ -40,15 +40,12 @@
         "vuex": "^3.6.2"
     },
     "devDependencies": {
+        "@babel/preset-env": "^7.16.11",
+        "@babel/register": "^7.17.0",
         "@intlify/eslint-plugin-vue-i18n": "^0.12.0",
         "@mdi/font": "^5.9.55",
-        "@vue/cli": "^4.5.15",
-        "@vue/cli-plugin-babel": "^4.5.15",
-        "@vue/cli-plugin-eslint": "^4.5.15",
-        "@vue/cli-plugin-router": "^4.5.15",
-        "@vue/cli-plugin-unit-mocha": "^4.5.15",
-        "@vue/cli-plugin-vuex": "^4.5.15",
-        "@vue/cli-service": "^4.5.15",
+        "@originjs/vite-plugin-commonjs": "^1.0.3",
+        "@originjs/vite-plugin-require-context": "^1.0.9",
         "@vue/test-utils": "^1.3.0",
         "@wikimedia/jsonschema-tools": "^0.10.4",
         "babel-eslint": "^10.1.0",
@@ -60,21 +57,26 @@
         "grunt-banana-checker": "^0.9.0",
         "mocha": "^8.4.0",
         "nyc": "^15.1.0",
+        "postcss": "^8.4.7",
+        "postcss-html": "^1.3.0",
         "rapidoc": "^9.1.3",
-        "sass": "^1.44.0",
+        "sass": "~1.32",
         "sass-loader": "^10.2.0",
         "sinon": "^11.1.2",
         "sinon-chai": "^3.7.0",
-        "stylelint": "^13.13.1",
-        "stylelint-config-wikimedia": "^0.11.1",
-        "vue-cli-plugin-vuetify": "^2.4.4",
+        "stylelint": "^14.5.3",
+        "stylelint-config-recommended-vue": "^1.3.0",
+        "stylelint-config-wikimedia": "^0.12.2",
+        "vite": "^2.8.4",
+        "vite-plugin-env-compatible": "^1.1.1",
+        "vite-plugin-test": "^0.0.5",
+        "vite-plugin-vue2": "^1.9.3",
         "vue-template-compiler": "^2.6.14",
-        "vuetify-loader": "^1.7.3",
-        "webpack-bundle-tracker": "^0.4.3"
+        "vuetify-loader": "^1.7.3"
     },
     "babel": {
         "presets": [
-            "@vue/cli-plugin-babel/preset"
+            "@babel/preset-env"
         ],
         "env": {
             "test": {
@@ -89,6 +91,11 @@
             }
         }
     },
+    "mocha": {
+        "require": [
+            "@babel/register"
+        ]
+    },
     "nyc": {
         "all": true,
         "extension": [
@@ -140,7 +147,8 @@
         "root": true,
         "parserOptions": {
             "parser": "babel-eslint",
-            "allowImportExportEverywhere": true
+            "allowImportExportEverywhere": true,
+            "sourceType": "module"
         },
         "extends": [
             "wikimedia/client-es6"
@@ -225,7 +233,7 @@
             },
             {
                 "files": [
-                    "vue.config.js"
+                    "vite.config.js"
                 ],
                 "extends": [
                     "wikimedia/server"
@@ -238,11 +246,13 @@
         "bin",
         "dist",
         "docs",
-        "toolhub/apps/toolinfo/data/language-data.json",
-        "vue/dist-tests/webpack-stats.json"
+        "toolhub/apps/toolinfo/data/language-data.json"
     ],
     "stylelint": {
-        "extends": "stylelint-config-wikimedia/grade-a",
+        "extends": [
+            "stylelint-config-wikimedia/grade-a",
+            "stylelint-config-html/vue"
+        ],
         "ignoreFiles": [
             "docs/_build/**/*.css",
             "vue/dist/**/*"
bd808 changed the task status from In Progress to Open.Mar 1 2022, 6:07 PM
bd808 removed bd808 as the assignee of this task.
bd808 moved this task from In Progress to Groomed/Ready on the Toolhub board.