Page MenuHomePhabricator

Migrate Developer Portal v2 prototype code to Gitlab production
Open, MediumPublic

Description

Migrate the Developer Portal v2 prototype from github to GitLab as production version. The prototype should not be seen as a copy-over artefact, but rather a reference to know what elements and behaviors to bring over to production with a more maintainable and sustainable structure. Scripts and production processes might be good ideas to bring over with some tidiness.

Migration status

Everything that needs to happen to move the prototype across. Ticked items are done and are merged.

NOTE: Some of those items can be canceled if we think we should not migrate those behaviors, change the way they're done completely, or rewrite them in the new site.
This list should be a helpful reference, not a prescriptive work order.

Pages and content

  • Markdown pages, one route handling every language
  • Basic page styling: headings, lists, tables, code, quotes, images
  • A page that isn't translated falls back to English, with a note to the reader
  • Error page, in the site shell and the reader's language
  • Richer page blocks: code tabs, callouts and alerts, highlight panels, cards, link rows, reusable snippets
  • Content for 40 of the 42 menu destinations (Talk to @apaskulin before moving content)

Page shell

  • Header with the Wikimedia mark and the site name
  • Top menu, built from one configuration file
  • Side menu showing the current section, which a page can switch off or swap
  • "On this page" list of the page's sections
  • Footer with the content licence and the policy links
  • Light and dark mode, with a control in the header
  • Right to left
  • Phone, tablet and desktop
  • Copyright line in the footer. Waiting on a decision about who it should credit.

Languages and translation

  • Interface text comes from translatewiki (74 messages, each documented for translators)
  • Catalogue of 575 languages, generated by a script we run by hand
  • A language is only offered once enough of it is translated
  • Separate thresholds for interface text and page text
  • A language that has qualified stays, even if its percentage later drops
  • Simple language picker in the header
  • Full language picker: type to search, shows each language's own name and its English name
  • Translated page content through translatewiki. Needs the changes the documentation team asked for first. (Talk to @apaskulin)
  • Confirm the repository is registered with translatewiki. Nothing in the repository does this, so no translations will arrive until it is.
  • Decide whether the threshold check stays in the build or moves to a script we run and review

API reference and explorer

  • API reference embedded and styled (Scalar)
  • The explorer itself: picking a wiki and a module (community api reference)
  • Enterprise version: bespoke widget lightly displaying Enterprise's OpenAPI spec
  • Wiki and module catalogues. A script sweeps the Wikimedia wiki fleet and saves what it finds as "source of truth" of available modules and which instance they are available on. The explorer can't work without it, and it's the biggest script in the prototype. It is also used for "deep linking" directly into the explorer, especially for "quick" links that do not include wiki instances (module-only links)
  • Links that open the explorer at one wiki, module and endpoint (deep linking)
  • Translated labels inside the API reference

Search
These are likely one of the bigger challenges given our breadth of language support and number of instances: ~800 instances, each with ~25 modules, each can be served in ~400 languages, and each module can have anywhere from 5 to 50 endpoints. There are explorations to be made here on how to resolve search (is it multilingual or English only, is it per-module or endpoint-specific, etc) and then see what product (@HCoplin-WMF ) decides for scoping, depending on available solutions.

  • Content search
  • Multilingual search
  • Endpoint search

Exposure
These require some exploration:

  • SEO support (sitemap? robots.txt? etc)
  • AI Optimization (llm.txt? static module catalog?)

Build and hosting

  • Every page built as static HTML
  • Automatic checks on every merge request: tests, types, translator documentation, generated files
  • Netlify preview for every merge request
  • Keep the _dev pages out of the production build
  • Make missing pages fail the build instead of just being counted
  • Run the static build in CI, not only the normal build
  • Decide where the site is hosted, and write down the deploy steps

Event Timeline

• bd808 triaged this task as Medium priority.Aug 22 2023, 7:43 PM

Declining as part of general task cleanup in preparation for Developer Portal version 2

Decided to move these tasks instead of declining for now. Sorry for the noise!

Mooeypoo subscribed.

I've started moving the interface and the basic behaviour from the prototype into the production repository, to get it ready for development.

I didn't copy any files across. I used the prototype as a reference for how things should behave and look, then wrote each piece again in the production site, keeping it small and easy to read.

I put a bit of emphasis on:

  • Keeping configuration separate from the code that uses it, and from the data (including generated data)
  • Keeping each piece clean and easy to maintain
  • Adding CSS variables, so design decisions are easy to change later

Writing it again took a bit more time, but the result is much smaller. A few examples:

  • "On this page" (the list of sections on a page) is about 120 lines here. In the prototype it's around 450. I take the list from data the page already has, instead of reading the finished page in the browser and watching it for changes.
  • The side menu doesn't need its own configuration. The same menu tree that draws the top menu also draws the side menu, so the site is only described once.
  • The language list comes from the Wikimedia API, through a script I run by hand, and the result is saved in the repository. The prototype had four languages typed in by hand. There are now 575, and nobody has to keep them up to date.

The point of all this is maintenance. Someone joining the project later should be able to open a file, see why it's built that way, and change one value in one place.

What's migrated

Pages and text

  • Markdown pages in any language, with one route handling all of them
  • Page styling (headings, lists, tables, code, quotes, images)
  • All interface text goes through a translation system (74 messages so far), and every one has a description written for translators
  • If a page exists in English but not in your language, you get the English version and a note saying why, instead of an error
  • An error page, in the site shell and in the reader's language

The page shell

  • Header with the Wikimedia mark and the site name
  • Top menu, built from one configuration file (42 destinations, 9 headings)
  • Side menu showing the pages in the section you're reading
  • "On this page" list of the page's own sections
  • Footer with the content licence, privacy policy, terms of use and code of conduct
  • Light and dark mode, with a control in the header to pick one
  • Language picker in the header, showing only the languages the site serves
  • Everything works right to left (try /he/)
  • Everything works on a phone, a tablet and a desktop

Under the surface

  • API reference embedded and styled (Scalar), at /_dev/api-reference
  • A catalogue of 575 languages, generated from the Wikimedia API by a script.
  • A language is only displayed in the UI list once enough of it is translated (threshholds that are defined in config variables). The percentage is counted on every build from the message files themselves, so adding a language just means putting its file in place. There's no list of languages to keep up to date.
  • Two separate thresholds, one for interface text (buttons, labels) and one for page text, so the two can be judged differently. Both are 75% for now. The interface one is working. The page text one is ready and waiting, because translated page content doesn't exist yet.
  • Once a language qualifies it stays, even if its percentage drops later. There's a note below on why that matters.
  • Every page is built as static HTML, so reading a page never waits for JavaScript
  • Automatic checks on every merge request: tests, types, translator documentation, generated files
  • A Netlify preview for every merge request, so you can look at the work without installing anything

NOTE: We should think whether the language threshhold consideration should be done on build (as it is right now) or maybe by a dedicated script pre-build so it can be reviewed and managed if/when languages are added or dropped. The current script is supposed to retain existing languages even if they dropped from threshhold, but depending on how often this happened, making this operation part of a pre-build process *might* help us control and validate for it better. This is just something to look for as we go.

Mooeypoo renamed this task from Migrate wikimedia/developer-portal to GitLab to Migrate Developer Portal prototype to Gitlab production version.Sep 14 2026, 7:02 PM
Mooeypoo updated the task description. (Show Details)
Mooeypoo updated the task description. (Show Details)

Note: Decision was officially made to *not* have to import any wiki pages into markdown for the prototype. I removed all the checkboxes that related to that, but just in case I missed any -- it is no longer a requirement.

Mooeypoo renamed this task from Migrate Developer Portal prototype to Gitlab production version to Migrate Developer Portal v2 prototype code to Gitlab production.Wed, Sep 16, 5:07 PM