Page MenuHomePhabricator

Simplify Add Link deployment process
Closed, ResolvedPublic2 Estimated Story PointsSpike

Description

Background & Problem

As part of T407818: Add a Link: Rollout "Add a Link" Structured Task to Chinese, Japanese, & Urdu Wikipedias, I started the preparation works for Add Link deployment to three new wikis. I attempted to follow https://wikitech.wikimedia.org/wiki/Add_Link#Enabling_on_a_new_wiki, and I noticed that docs page is: describing a fairly complicated (and manual) process, as well as being heavily out of date (there is not a single line in that section that is 100% accurate at this point).

Currently, the documentation page ignores the introduction of those key components:

  • MediaWiki-extensions-CommunityConfiguration the extension (extensions/GrowthExperiments/maintenance/changeWikiConfig.php no longer exists)
    • CommunityConfiguration:ChangeWikiConfig.php uses a different syntax for changing the wiki-page (the three arguments at the end are PROVIDER, KEY and VALUE; --json no longer exists)
    • link-recommendation task type is configured under link_recommendation (which the docs page does not reflect)
  • mwscript is no longer used to execute maint. scripts; instead, we use mwscript-k8s

Converting the excluded section population from Add Link took a good while, running into mwscript-k8s confusion (T410451) and missing all the changes in syntax that happened in the meantime. A working version of the command line is at T407818#11385945.

In addition to the documentation being out of date, it is not a sustainable way to deploy new features. We should not require ourselves to run a bunch of manual steps that are fragile, as illustrated above.

Options

There are three approaches we could take here:

  1. Create a scraffolding script like initAddLinkConfig which would do the manual steps for it
  2. Stop pre-populating excluded sections configuration on per-wiki basis (this would also solve T345562 by declining it)
  3. Merely update the documentation page to reflect the changes that happened in the meantime

If we want to keep the current behaviour, option 1 is (by far) the most preferred. However, it would require a bit of work, as we need to figure out how to access wiki_sections.jsonl in a programatic way (downloading F35092312 from a maintenance script doesn't seem to be a good idea). If we're comfortable with leaving excluded sections to the wikis, then option 2 would make a lot of sense to me.

At a minimum, we should update the documentation page to be truthful. However, this alone is not solving the unnecessary complexity bits the deployment process currently has.

Task Scope

Within this task, we should:

  • decide on what route we want to take,
  • fill relevant subtasks
Related Tasks

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
KStoller-WMF added subscribers: KStoller-WMF, Michael.

Moving to Current Quarter, as I think it would be ideal if we complete this task along with the work to rollout Add a Link to remaining Wikipedias (that have V2 model support).

Michael changed the subtype of this task from "Task" to "Spike".Nov 26 2025, 3:43 PM
Purpose of the manual

The current manual has three CC-facing steps:

  1. Pre-create a boilerplate configuration (by saving { "type": "link-recommendation", "group": "easy" } under link-recommendation if the CC page does not exist)
  2. Generate excluded sections using a pre-computed wiki_sections.jsonl (T306792#7897336, F35092312)
  3. Save that list of excluded sections onto the config page

Once this is completed, the manual sees to first set wgGENewcomerTasksLinkRecommendationsEnabled to true, followed by wgGELinkRecommendationsFrontendEnabled. This is still accurate.

Differences
  • Using a CC1.0 script (extensions/GrowthExperiments/maintenance/changeWikiConfig.php no longer exists`)
    • today's equivalent: CommunityConfiguration:ChangeWikiConfig
    • the page title is no longer an identifier; instead, the provider name should be used (GrowthSuggestedEdits here)
    • --json parameter is not recognised; all values are instead parsed as JSON, so this just needs to be excluded
    • --create-only parameter is not recognised; it is not possible to run ChangeWikiConfig so that it only does a first edit; I believe this is not necessary, see below
  • link-recommendation was replaced by link_recommendation instead
Open Questions

Really only one: How important are excluded sections for us?

Option (A): Keeping them

We have a long-standing task (T345562: Generate excludedSections list for wikis missing in wiki_sections.jsonl) about generating the excluded sections for all wikis. The main issue is that the section alingment data we used for this is from 2022, and many wikis were created since then. Research project documentation exists as well as the source code, so we should be in principle able to repeat that work. However, it would be working in not-so-familiar area for us, likely resulting in more time spent.

Option (B): No longer provisioning excluded sections automatically

The only alternative besides this is to remove excluded sections pre-generation from the docs, since it is not going to be successful for the vast majority of new deployments (=they are going to happen on new-ish Wikipedias).

This is probably a question for @KStoller-WMF. Unfortunately, as I'm not very familiar with the section-alignment code, I'm not sure how easy it would be to run. If we allocate an engineer-sprint to find that out, we might know more.

If we didn't have so many competing high priorities, Option A would be ideal, but I personally think our engineering time would be better spent tackling other problems and so I suggest moving forward with Option B.

Option (B): No longer provisioning excluded sections automatically

Communities have the ability to adjust excluded sections via Community Configuration, and edit tags will point experienced editors to a MW page that explains:

Newcomer tasks are configurable by communities by visiting Special:CommunityConfiguration locally.

It's not perfect, but we are providing communities with the tools to improve the task, and I think that's good enough for now.

I updated most of the documentation based on @KStoller-WMF's answer (thank you!). I still need to update the command to prepopulate sections (mostly to keep archives happy), and then this task can be considered resolved.

Urbanecm_WMF added a subscriber: Sgs.

And adjusted the comment, explicitly framing excluded sections as optional. I'd call that simplified enough. Moving to Code Review, mostly to get a perspective from @Michael and/or @Sgs (or other engineers, if they have any).

And adjusted the comment, explicitly framing excluded sections as optional. I'd call that simplified enough. Moving to Code Review, mostly to get a perspective from @Michael and/or @Sgs (or other engineers, if they have any).

I can't fully verify the script snippet because I'm not a deployer and so not familiar with the k8s-related details, but the rest of that sections reads very straightforward and matches my understanding about what needs doing. Thank you!

IIRC, that script should be runnable with restricted permissions (which you have)? In any case, thank you for the review!