Closing as duplicate of: https://phabricator.wikimedia.org/T376607

Description

We perform end-to-end testing as part of the CI pipeline process using Mocha. For every endpoint in the MediaWiki REST API, we need to add a check that verifies the returned response matches the public API specifications. This will allow us to automatically detect, capture, and surface instances where the endpoint behavior does not match what is documented in the response body, which increases API stability, improving the MW API developer experience.

Problem statement

The main defense that we have against spec deviation is ensuring that undocumented changes don’t make it into production in the first place. We need to catch variation in the spec vs actual endpoint behavior as part of the release process as a way to ensure documentation accuracy and prevent accidental changes. By introducing spec validation as part of end-to-end testing for every endpoint, we can improve stability for our API.

Conditions of acceptance

• Update Mocha tests to utilize the reusable methods for fetching JSON schema response and comparing it against the php response using the validation method.
• Include spec validation testing to all of the ‘page’ endpoints.
  • POST: w/rest.php/v1/page
  • GET: w/rest.php/v1/page/{title}
  • PUT: w/rest.php/v1/page/{title}
• Perform updates that force the test to fail; verify that it will trigger a CI error.
• Verify that the test works across different HTTP responses for each endpoint.
• Demonstrate this capability to the team. This is the final work that needs to be done for the ‘page’ endpoints!! 🎉

Dependencies

Assumes the following work is complete:

• https://phabricator.wikimedia.org/T375530
• https://phabricator.wikimedia.org/T376326
• https://phabricator.wikimedia.org/T376339

Out of scope

This work only focuses on the ‘page’ endpoints. We will have subsequent stories for additional endpoints.