Page MenuHomePhabricator

Create Sitemap API Module
Open, HighPublic2 Estimated Story Points

Description

Description

Sitemap endpoints were recently introduced to MediaWiki Core. They are already following our module based versioning structure, and can safely be allocated into their own module. Creating a module will allow these endpoints to be independently managed and versioned.

Conditions of acceptance

  • Sitemap endpoints are broken into their own API module.
  • The Sitemap Module is not included in the REST Sandbox.
    • Verify the Sitemap endpoints are no longer visible, and the module is not present in the dropdown.
NOTE: We do not need to lint nor improve the spec as part of this story, as this specific module will be suppressed from the spec generation process.

Event Timeline

HCoplin-WMF renamed this task from Create Sitemap Module to Create Sitemap API Module.Fri, Nov 7, 2:36 AM
HCoplin-WMF updated the task description. (Show Details)

The task description says two different things regarding suppression/visibility:

The Sitemap Module is not included in the REST Sandbox.

That part is easy. We just don't include the Sitemap module in the $wgRestSandboxSpecs config variable. The endpoints will still work fine.

We do not need to lint nor improve the spec as part of this story, as this specific module will be suppressed from the spec generation process.

That sounds different. Currently, all enabled modules will be listed in the discovery endpoint:

https://en.wikipedia.org/w/rest.php/specs/v0/discovery

Do we need to suppress the new site module from this as well? If so, we'll need to add that capability. That shouldn't be a huge effort, mostly we'd just need to agree how it works. I can post a proposal inspired by from the beta module experimentation if that would be helpful.

Per discussion, we prefer spec generation to be suppressed as well. I had forgotten about T409517: REST: API modules can be suppressed/opt-out of spec generation

HCoplin-WMF set the point value for this task to 2.

Change #1208429 had a related patch set uploaded (by BPirkle; author: BPirkle):

[mediawiki/core@master] REST: add "site" module, contining "sitemap" endpoints

https://gerrit.wikimedia.org/r/1208429

Sitemaps are currently at urls like:
https://en.wikipedia.org/w/rest.php/site/v1/sitemap/0

The initial patch uses:
module file name: site.v1.json
module id: site/v1
module title: Site API
module description: Information about the site as a whole, such as sitemaps

This is all easy to change if it isn't what we want.

I left endpoint descriptions/summaries and other improvements to T402691: [5.2.5 Milestone] Improve sitemap API documentation

Keep in mind that modules are versioned units. If we put other endpoints pertaining to the site as a whole (not sure what, maybe status, health, navigation, etc.) into the site.v1 module, and later decide to make changes to any of those endpoints that require a version change, then that will introduce a new version for all of them. (Multiple versions of a module can coexist. Whether that's a good idea or not, and for how long, is something we can decide later.)

Also keep in mind that spec/sandbox/explorer visibility is on a module-level granularity. That means if we suppress display of the sitemap endpoints in the sandbox now (and exposure of their spec in the /discovery endpoint), then any other endpoints we later add to the site/v1 module will be likewise hidden/suppressed.

I don't object to site/v1, I'm just mentioning all this to make sure that we get used to including it in our thought process as we add modules.

Assuming we continue as described in the task, here's the rollout plan:

  • review/refine/merge the attached patch
    • This will add the site/v1 module to the codebase
    • Because the new module isn't activated in config, it won't do anything (even after the train rolls)
    • The sitemap endpoints will continue to be happily served via the coreRoutes.json entry
  • wait for the train to deploy the patch to all wikis
  • create and deploy a config change to activate the site/v1 module
    • The sitemap endpoints will immediately, and without error, start being served by the site/v1 module entries rather than the coreRoutes.json entries
    • The REST Sandbox will continue to display the sitemap endpoints under "MediaWiki REST API", because they're still included in coreRoutes.json
    • "Try it out" will continue to work for the endpoints (because the urls match)
    • The site/v1 module will not appear in the REST Sandbox (because we won't add it to $wgRestSandboxSpecs)
    • The site/v1 module will appear in the /discovery endpoint response under the specs/v0 module. But at least that's comparatively obscure.
  • create and merge a core patch to remove the sitemap endpoints from coreRoutes.json
    • As this rides the train, the sitemap endpoints will disappear from the REST Sandbox on each wiki group
  • implement and deploy T409517: REST: API modules can be suppressed/opt-out of spec generation, to remove the site/v1 module from being discoverable, even via the /discovery endpoint. (This could ride the same train as the change to remove the sitemap endpoints from coreRoutes.json, if it is ready in time.)

Note: we can't deploy a config change to activate a module that doesn't exist (well, we could but it'd cause errors, so let's not). We could change that to be a silent failure, and shorten the rollout plan to one train instead of two. But we probably shouldn't. That seems like a failure we'd generally want to know about and the cost of doing this particular rollout across two trains doesn't seem all that high.

Change #1208429 merged by jenkins-bot:

[mediawiki/core@master] REST: add "site" module, contining "sitemap" endpoints

https://gerrit.wikimedia.org/r/1208429

Change #1213515 had a related patch set uploaded (by BPirkle; author: BPirkle):

[operations/mediawiki-config@master] REST: enable the site.v1 module

https://gerrit.wikimedia.org/r/1213515

Change #1213518 had a related patch set uploaded (by BPirkle; author: BPirkle):

[mediawiki/core@master] REST: remove sitemap endpoints from flat routes in coreRoutes.json

https://gerrit.wikimedia.org/r/1213518

Change #1213515 merged by jenkins-bot:

[operations/mediawiki-config@master] REST: enable the site.v1 module

https://gerrit.wikimedia.org/r/1213515

Mentioned in SAL (#wikimedia-operations) [2025-12-01T21:13:00Z] <cscott@deploy2002> Started scap sync-world: Backport for [[gerrit:1212670|Deploy Parsoid Read Views to 19 wikis (T411283)]], [[gerrit:1213497|Change the README to Markdown]], [[gerrit:1213498|noc: Point links in /conf to Gitiles rather than Differential]], [[gerrit:1213515|REST: enable the site.v1 module (T409516)]], [[gerrit:1213559|cirrus: Apply increased near match weight on commonswiki (T408154)]]

Mentioned in SAL (#wikimedia-operations) [2025-12-01T21:16:57Z] <cscott@deploy2002> cscott, ebernhardson, tgr, arlolra, bpirkle: Backport for [[gerrit:1212670|Deploy Parsoid Read Views to 19 wikis (T411283)]], [[gerrit:1213497|Change the README to Markdown]], [[gerrit:1213498|noc: Point links in /conf to Gitiles rather than Differential]], [[gerrit:1213515|REST: enable the site.v1 module (T409516)]], [[gerrit:1213559|cirrus: Apply increased near match weight on commonswiki (T408154

Mentioned in SAL (#wikimedia-operations) [2025-12-01T21:25:09Z] <cscott@deploy2002> Finished scap sync-world: Backport for [[gerrit:1212670|Deploy Parsoid Read Views to 19 wikis (T411283)]], [[gerrit:1213497|Change the README to Markdown]], [[gerrit:1213498|noc: Point links in /conf to Gitiles rather than Differential]], [[gerrit:1213515|REST: enable the site.v1 module (T409516)]], [[gerrit:1213559|cirrus: Apply increased near match weight on commonswiki (T408154)]] (duration: 12m

Change #1213518 merged by jenkins-bot:

[mediawiki/core@master] REST: remove sitemap endpoints from flat routes in coreRoutes.json

https://gerrit.wikimedia.org/r/1213518