Page MenuHomePhabricator

Migrate away from unmaintained dependencies in portals' codebase
Open, Needs TriagePublic

Description

After some discussion with @Jdrewniak, I was aware that the gulp-based build steps in portals' codebase "depends on a web of unmaintained plugins", and should be completely rewritten.

Possible alternative of gulp:

  • grunt: Used a lot in Wikimedia projects
  • Use vanilla NodeJS scripts

Possible alternative of gulp-useref and other tools to build portal html:

  • rollup
  • vite (Used in Codex)

Event Timeline

Diskdance renamed this task from Clean up Wikimedia Portals' code to Migrate away from unmaintained dependencies in portals' codebase.Aug 13 2022, 2:37 AM
Diskdance removed Diskdance as the assignee of this task.
Diskdance updated the task description. (Show Details)
Diskdance added a subscriber: Jdrewniak.

I know @Jdforrester-WMF did some work and removed grunt pieces from portals, more can be done, simply move them to the nodejs vanilla. I'd be more than happy to review them but I can't find time to do it myself :(

Actually Jan did a bunch of the work too. :-) I'd also be happy to help with that.

We could probably migrate to Vite and Vue and use SSG to generate a static site.

Following up here from your post on Wikitech-l. Speaking from the perspective of the Design-System-Team, which is developing Codex and driving adoption of Vue in the Wikimedia ecosystem, this sounds like a great project. There are a few different ways this could be accomplished with Codex, which someone more technical can elaborate on. I think the search component will be the tricky part.

However, the fact that Portals remains unmaintained but works perfectly fine makes this change risky in my view. It sounds like the core issue here is around unmaintained dependencies, which don't necessitate a full rewrite to solve. Are there other motivations for rewriting everything? The value-to-effort ratio seems low at this point, although I can see this being a good hack project where we'd be comfortable throwing away the code if it doesn't actually end up better (in this case "better" defined as having a unified design and being more maintainable without sacrificing performance).

We're happy to discuss providing design and code review support if someone does move forward with this though.

We could probably migrate to Vite and Vue and use SSG to generate a static site.

With regards to moving to a different build-tool/Vue SSR, I think that would be difficult at this stage because the codebase lacks some clear boundaries between the data-aggregation and rendering layer. For example, the client-side translation files are written to disk within the Handlebars.js controller file, and numeral formatting also happens within a Handlebars.js helper.

If we had better separation between data & rendering, where it was a matter of piping a predefined JSON object into some templating language/framework, then switching out that rendering layer to something entirely different would be more feasible.

Personally though, I think that a lot of modern build tools incur a very large and unnecessary Javascript payload (I'm heartened by tools like Astro though, that are actually capable of producing HTML output with zero JS). Currently, the portals ship about 8.5kB of JS (gzipped) and I wouldn't want performance to suffer with of a change in tooling.

To @CCiufo-WMF's point though, I think moving as much of the code to Codex components would be great for long-term maintenance. I think we could start with looking at Codex css-only components first, since introducing Vue/Vite would most like require introducing new tooling for performance considerations (how big is the Codex typeahead search component if we bundle it with Rollup.js and use tree-shaking?).

With regards to the Gulp Plugin Graveyard, there are ways we can reduce that in a gradual way. We can remove some tasks entirely like the sprite-svg task, which provides SVG fallback images, and we can rewrite a lot of the tasks to Gulp inline plugins which are (almost) vanilla Node.js scripts that are based on the Node streaming API. Even if we were to move to something like Vite, I still think Gulp could be useful for the data-related task orchestration, but essentially, I think we should be careful not to trade in a bunch of unmaintained plugins for a different set of yet-to-be unmaintained plugins.