Background

Spec linting is a key requirement for enforcing API standards in a scalable way across distributed API owning teams. Specifically, linting provides a programmatic way to ensure quality and consistency across API specifications, which improves adoption of standards defined by a centralized governance team. These types of tools are easier for teams to adopt than simply reading a style guide, and offer a direct means of enforcement by including it as a required step in the deployment process. This approach moves the detection earlier in process, and ensures that no APIs are released that do not meet our quality or consistency standards for the spec itself.

Having a high quality spec, in turn, is a foundational element for downstream activities related to additional automated testing and enforcement, such as validating that the spec matches actual API endpoint behaviors. Similarly, it ensures that we deliver a higher quality experience for internal and external developers looking to use our APIs.

Problem statement
Many web APIs exist across Wikimedia, most of which were developed without consideration for standardized tooling and style. We are transitioning to an API development model where we will have a centralized governance team overseeing distributed ownership of bespoke APIs. API authors need guidance for creating good API documentation through their OpenAPI spec to create a consistent developer experience. Additionally, although there have been some attempts from the Tech Writer team to provide style guides and other documented standards, but we have seen limited and inconsistent adoption based on when and where the APIs were authored. We also assume that many developers (or just humans) will not read and fully internalize such style guides; instead, we must meet our API developers where they already live (in-code) and ensure that our standards can be programmatically enforced so that teams cannot ignore or work around them when creating or updating endpoints.

Linting provides a means of creating centralized rules, which can then be programmatically enforced, for all REST APIs with defined OpenAPI specs.

Impacted users
The primary users of the linting service will be Wikimedia REST API authors, as OpenAPI and the spectral framework only applies to REST APIs. This is because linting opens the door for making it easier to know how to properly document an API, and moves that guidance earlier into the development process by using the linting tools on demand to get clear guidance as part of spec creation and endpoint development, instead of creating manual docs on-wiki after the endpoints are released. This integrates the documentation process with API and endpoint development more natively, with the long term horizon including integrated evaluation that prevents APIs from being released without proper documentation.

The secondary beneficiaries is the API users themselves. Spec linting inherently improves the consistency and quality of the specs, which results in a more consistent and complete experience for our technical volunteers, staff, researchers, and other API consumers.

Scope

• Spec linting via Spectral
  • Create a basic spec linting ruleset (recommended: spectral) based on tech writer defined Open API style guide 
  • Once complete, publish Wikimedia ruleset to Spectral repo
• Create a means of linting a spec on demand
  • Assume users have access to a locally stored or manually copied spec; we do not need to generate the specs on demand.
• [Stretch] Make the linting tool available through Toolforge, with a light UI, so that any Wikimedian or third party MediaWiki user can use it to lint their spec for Wikimedia API standards.
  • UI is built using Codex
  • Users are able to link to an endpoint, paste a block of text, and/or upload a file to provide the spec
  • The UI reports relevant errors with context about the rule so that readers can understand it
  • Users may browse the ruleset and/or style guide for more context
• [Stretch] Determine the best place for linting enforcement (eg: CI, some other automated test suite) and socialize decision across MediaWiki organization.

Out of scope

• Automated linting as part of CI or other automated testing is considered out of scope for this body of work
• Translation linting — this work exclusively focuses on OpenAPI specs. We will assume English as the only spec language for linting. We will look into enforcing spec translation coverage and structures (such as templates to guide wikitranslate activities) at a later date.
• On-demand spec generation.

Expected Outcomes

Spec linting will ensure that all Wikimedia APIs adhere to the common standards and styles that we define, so that we can create higher quality specs, which are the backbone of a better developer experience. We will have an easy to use, self-service experience that any developer within Wikimedia or our broader volunteer and third party ecosystems can use. We will also create a publicly published version of our ruleset as a means of participating in the broader OpenAPI open source community, and provide a means of volunteers reviewing, providing feedback, and potentially contributing to the ruleset directly.

Introducing a centralized linter will also enable the transition to a centralized governance model to oversee and standardize API development across Wikimedia. MWI will be responsible for defining and managing the ruleset, which will then be used by teams across the organization to ensure that all of our OpenAPI specs have a consistent level of quality, depth, and tone. This will be the first real/functional instance of this model.

Known risks & limitations

We are currently in a bit of "a chicken and an egg" situation — until we have better adoption of our spec standards, we cannot begin globally enforcing them; yet, until we begin global enforcement, we cannot guarantee adoption of our standards. We plan to address this issue by partnering with early adopter teams to work out kinks and improve the experience, then implement integration with CI or other automated testing in a future quarter, after we've hit critical mass of compliance and coverage for existing APIs.

Enforcement also becomes more difficult as we look to volunteer maintained extensions. While there are not many, some do exist, and we must engage with the owning volunteers to follow our guidelines. We believe that the self-service Toolforge interface will help with this.

• 1. Related Work
• OpenAPI style guide: The TechWriter team is responsible for creating and documenting a style guide for OpenAPI spec generation. While an initial version exists, it is still in its infancy and will be subject to both expansion and refinement over time. As the style guide evolves, so too must the linting ruleset.
• MediaWiki API Handler Refactoring: Goal of the project is to decouple spec generation and other handler activities. This will improve clarity and API user experiences for spec generation. The ideal outcome is that generating specs on demand will become easier, and what API publishers need to do to fully generate their spec becomes clearer. This will (hopefully) increase adoption of OpenAPI specs across the board.

Dependencies

Globally enforcing linting requires additional adoption of our core mechanisms and requirements for spec generation so that we don't erroneously block CI releases while we close the gap.

Next steps

• Integrate API linting into existing Wikimedia processes, so that linting occurs automatically. Possible options include:
  • CI testing
  • Local automated testing
  • Maintenance scripts
• Partner with at least one additional API owning team to begin improving their API specifications to meet our standards, including using the new linting tool
• Improve translation experience & create custom translation "linting" process as part of spec generation
  • Update QQQ quality/templating to support wikitranslate 

Figure out if there is a good way to ‘lint’ translation as part of spec generation (NOTE: Banana Lint might be an option here? But might only check if QQQ exists; not what the content or conventions are)

• Specific problem to solve: QQQ definitions are bad and require more context for translators; we likely need templates. https://phabricator.wikimedia.org/T404120 

Investigate a similar linting approach for the Action API.

Deadline

This experiment is expected to last through FY2526 Q2 (Sept-Dec). Because of the end of the year holidays and MediaWiki offsite, we expect work to wrap up by the end of November.