5.2.5b [Productionalize API spec linting]: If we finalize the linter architecture and a core set of linting rules, we can improve the quality of OpenAPI descriptions and start introducing programmatic guarantees of their compliance into CI workflows in API development processes.

Hypothesis status: Complete

Hypothesis outcomes

The hypothesis was successful. We finalized a set of OpenAPI linter rules and the architecture of linting tools, which allowed us to improve the MediaWiki REST Framework, the MediaWiki REST API OpenAPI description (OAD, or "spec"), and add the linter to CI.

OpenAPI linter:

• We finalized a set of 72 linting rules designed to ensure that Wikimedia OpenAPI descriptions contain all the relevant information in a correct format.
• We improved the architecture of the linter by splitting it into two codebases - separating the ruleset from its user-facing representation. While we worked on the former, we also made improvements to the UI of the latter.
• We created an NPM package for the linter ruleset to allow it to be reliably used in automated pipelines. This has the added benefit of making the ruleset easier to include in other tooling and local linters, as the package is freely and publicly available.

MediaWiki REST Framework and API description:

• We fixed over 150 issues discovered by the linter in the MediaWiki REST API OAD, in endpoints directly maintained by the MediaWiki Interfaces team.
• We added security mechanism information to the MediaWiki REST API OAD.

Linting in CI:

• We added linting to the api-testing CI job. This will allow us to verify the MediaWiki REST API description automatically for every submitted patch.

Documentation:

• We completed the initial version of the OpenAPI style guide.
• We documented all the rules implemented in the linter in a dedicated linter rule reference.
• We added important information for contributors to the ruleset DEVELOPERS.md file to ensure consistency and maintainability of all OpenAPI-related resources.
• We integrated a dedicated OAD example with linter tests to ensure API maintainers can always access a reference OpenAPI description that passes all linter checks.

Background

OpenAPI description (OAD, commonly referred to as a "spec") is a formal description of an API written in compliance with the OpenAPI specification. OADs serve many purposes: documentation, testing and validation, API client and server generation, and others.

As part of the broader API strategy, we're transitioning to using OpenAPI descriptions as the main source of API reference documentation. Well maintained, high quality OADs will become a required artifact for all Wikimedia APIs. To ease this transition and to ensure that creating high quality OADs is a straightforward process, we have created a number of different resources, including an experimental version of a spec linter developed in hypothesis 5.2.5. See that hypothesis (T406171) for more information on why we chose linting as the main mechanism for ensuring OAD quality.

The spec linter is a web-based tool that accepts a valid OpenAPI description (version 3.0.0) and returns a list of potential problems that would make the description incomplete or inconsistent. The current version fulfills the earlier requirements but doesn't yet implement all the planned linting rules and isn't ready for deployment in CI workflows.

This hypothesis seeks to finalize the linter so that API developers inside and outside the Foundation can use it to create complete, correct, and compliant OpenAPI descriptions. We will also prove that the linter works as a convenient and useful tool for making iterative improvements to existing OADs, and that - after some modifications - it's possible to embed it in CI workflows and thus in regular API development processes.

Problem statement

As we're transitioning to an API development model with centralized governance and distributed ownership, there's a need for flexible tools and resources that will help ensure consistency in our API offering without introducing unnecessary process overhead. A linter with a centrally-managed set of rules, compatible with different development processes via straightforward CI integration seems like the right solution to this problem.

Impacted users

The primary impacted group are Wikimedia API developers and maintainers. These users are directly affected by the process changes and will benefit from the new tooling developed in this hypothesis.

The finalization of the linter will indirectly impact API users, who should benefit from higher quality documentation of our APIs.

Scope

• Experiment to document security mechanisms as part of the OAD
• Implement the complete set of linting rules
• Create the linter library/package
• Create a proof of concept of linting as part of a CI workflow
• Ensure compliance of the MediaWiki REST API OAD with linting rules
• Finalize the maintenance process for the example OAD

Out of Scope

• This work does not include enforcing linting requirements for OADs beyond the MediaWiki APIs owned directly by the MediaWiki Interfaces team.

Expected Outcomes

• The spec linter implements the full set of agreed-upon rules and can check OpenAPI descriptions for compliance with these rules.
• The spec linter returns no errors or warnings when linting the OpenAPI description of the MediaWiki REST API.
• The OAD for the MediaWiki REST API features information about at least two of the supported API security mechanisms.
• A proof of concept exists for including the spec linter in a CI workflow (most likely starting with a subset of core MediaWiki REST API endpoints). This proof of concept can be used as an example for how to integrate the linter in other workflows.
• The maintenance process for the OAD example is known and documented.

Known risks & limitations

• The current set of agreed-upon linting rules doesn't fully consider the technical capabilities of libraries used to implement the spec linter. It's possible that implementation of some rules might prove more complex and time-consuming than anticipated.
  • We will attempt to identify such cases early, perform research spikes to determine viability, and remain flexible in what we include in the final set of linting rules.
• The size of the MediaWiki REST API OAD and the complexity of its generation mechanism might make it impossible to resolve all linting errors and warnings in this hypothesis.
  • If this happens, we will file appropriate follow-up tasks detailing how we intend to resolve these problems in the future - as part of another hypothesis or essential work. Solutions might include: reevaluating the rules included in the final set, making changes to the OAD generation mechanism, adding or improving descriptions in the OAD.
• The MediaWiki REST API supports many different security mechanisms. It's possible that the OpenAPI description format isn't suitable for describing these mechanisms completely and correctly.
  • If that's the case, we will focus on including the most important security mechanisms in the OpenAPI description while documenting other mechanisms elsewhere.

Dependencies

Work in this hypothesis is a prerequisite for broader validation and enforcement of OpenAPI style rules in OADs for all Wikimedia APIs.