tl;dr: add "audience designations" to REST modules. These are strings that appear in module ids and endpoint urls. Each audience designation is associated with a specific set of functionality and expectations. To handle special cases, this functionality can be overridden on a per-module basis.

See T413443: [5.2.8 Epic] Include Audience designations on modules for additional discussion and references.

Purpose

REST modules have several behaviors, and will soon receive more, governing their availability and how they are revealed to callers:

• is the module enabled or not? (exists now, but details will change)
• can an OpenAPI spec be generated for the module? (not currently controllable)
• is the module listed in the /discovery endpoint of the specs module? (not currently controllable)
• is the module visible in the REST Sandbox (exists now, but details will change)
• is the module visually distinguished in the REST Sandbox to let callers know it has special considerations (beta, etc.) (does not yet exist)

User research has shown that:

• our current systems for managing these are cumbersome
• the most common case (module is enabled and published to the REST Sandbox) is not automatic
• it would be helpful to allow module developers to control these aspects of their module without requiring config changes
• it would be helpful to callers to more easily understand some of these behaviors and expectations

Audience Designations accomplish this by adding a string to the REST module id, which appears in the url. The MW REST API framework already supports one such string: "beta". This is currently used for the Attribution API, which supports calls like this:
https://test.wikipedia.org/w/rest.php/attribution/v0-beta/pages/Earth/signals?redirect=true

Notice the "-beta" after the version string. That's the audience designation.

Available Audience Designations

The audience designation concept is extensible. However, we want to avoid a confusing number of audience designations. Initially, two audience designations will be supported:

• beta (already exists in a limited way)
• internal (not yet allowed)

We anticipate adding a "restricted" designation in the near future. Details of its behavior are still being determined and will be documented in a separate task.

Behaviors

The functionality, conventions, and expectations for each of these designations are:

• no designation supplied (the most common case)
  • endpoints are publicly available
  • changes and removal are handled through predictable versioning and deprecation processes
  • generated OpenAPI spec is publicly available
  • the module is listed in the REST Sandbox
  • a current example on production is specs.v0 aka the Specs API

• beta
  • endpoints are publicly available
  • no stability guarantees apply. Changes or removal may occur at any time
  • beta endpoints are expected to be short-lived. They should either be sunset or changed to a non-beta version.
  • generated OpenAPI spec is publicly available
  • the module is visually distinguished in the REST Sandbox. Opt-in is required to view.

• internal
  • endpoints are publicly available
  • stability is based on internal needs. External callers should make no assumptions about stability.
  • generated OpenAPI spec is publicly available
  • the module is visually distinguished in the REST Sandbox. Opt-in is required to view.

Functionality Requirements

We anticipate needing the following functionality, either now or in the near future:

• disabled (endpoints are not available to be called, spec is not viewable or discoverable, no sandbox entry)
• hidden (endpoints are callable but spec is not accessible or listed in /discovery, no sandbox entry)
  • spec should still be available within tests, so that things like response schemas can be used to validate endpoint behavior
  • because our code is open source, callers can and will still become aware of the existence of all modules. Spec suppression is intended to communicate that a module should not be publicly used, not to prevent public knowledge of its existence.
• discoverable (endpoints are callable, spec is listed in /discovery, no sandbox entry)
• toggleable/opt-in: (endpoints are callable, spec is listed in /discovery, opt-in to view on sandbox)
  • there will be multiple opt-in groups. See the related subtask for details.
• published (endpoints are callable, spec is listed in /discovery, sandbox entry)

Not all of these abilities currently exist in code, and the ones that do will need to be adjusted so that they recognize Audience Designations, and can be overridden by the new RestModuleOverrides configuration variable (see below).

Layering

Because audience designations are new and we're still learning exactly how they'll work, we should avoid tightly tying any particular designation to any particular behavior. A layer in the code should translate the behaviors of designations to functionalities. Audience designations should be mapped to the various types of functionality in a way that lets us easily make adjustments. For example, if it is determined that specs for -internal modules should be completely hidden from the REST Sandbox, the associated code change should be minimal. This mapping should be hard-coded and not exposed via configuration.

The mapping looks like:

Some functionality sets (disabled, hidden, discoverable) don't map to one of the initial audience designations. We should still implement them, for use in the RestModuleOverrides config variable (see below).

Configuration

We currently use two config variables to control behavior of REST modules:

• RestSandboxSpecs
• RestAPIAdditionalRouteFiles

Because these are currently in use, we should continue to support them for a transition period. Any modules currently referenced by these config variables should continue to behave the same way they do now.

• Minor exception: existing "beta" modules should move into a toggleable section in the REST Sandbox
• RestSandboxSpecs will continue to be needed for specs outside MediaWiki (such as the RESTbase endpoints). We expect to eventually handle those differently, but that's outside the scope of the Audience Designation work

To handle special cases, a new configuration variable, RestModuleOverrides, will allow assigning functionality sets on a per-module/per-wiki basis.

'wgRestModuleOverrides' => [
	'default' => [
		'mymodule.v1' => [ 'mode' => 'disabled' ],
		'myothermodule.v2' => [ 'mode' => 'discoverable' ],
		‘site.v1’ => [ 'mode' => ‘hidden’ ]
	],
	'+testwiki' => [
		'mymodule.v1' => [ 'mode' => 'published' ]
	],
],

Notice in the above example that enabling a module on only a subset of wikis means setting it to 'disabled' by default, then selectively enabling it on the desired wikis. For the common case of REST APIs in extensions where the REST module should be enabled everywhere the extension is enabled, this will be unnecessary - if the extension isn't enabled on a wiki, there's no REST module to disable. But in other cases, such as modules in core that should only be enabled on some wikis, or for extensions that expose REST modules that should only appear on some wikis (ex. WikimediaCustomizations), this pattern will be needed.

The array_replace_recursive merge strategy will be needed to make this pattern work. See the PasswordPolicy variable for an example.

REST Sandbox Opt-in

For modules using the "opt-in" functionality set (beta and internal by default, or any module specified that way in RestModuleOverrides), the user must take proactive action to see the module in the REST Sandbox.

The exact visual details are left to that implementation task. There should be multiple opt-in groups. At a minimum, beta and internal modules should be toggleable separately, but the set of groups should be open-ended, and it should be possible to specify new groups via config.

Additionally, the RestModuleOverrides config value will, in this case, need to allow configuration of which opt-in group the module belongs to. Those details are also left to that implementation task. If reasonably possible, it would be desirable to treat opt-in groups as dynamic, appearing in the REST Sandbox simply by being assigned to a module, then requiring them to be specified in a separate config variable.

Implementation Order

Tasks should be performed in the following order:

• T422754: REST: Audience Designations - functionality sets
• T422756: REST: Audience Designations - add RestModuleOverrides config value
• T422770: REST: Audience Designations - clean up module enabling (including its subtasks)
• T422771: REST: Audience Designations - publish modules to REST Sandbox by default
• T422870: REST: Audience Designations - add "internal" designation
• T422873: REST: Audience Designations - Visually separate "opt-in" modules in the REST Sandbox