Page MenuHomePhabricator

Reading List REST Interface: testing plan
Open, MediumPublic

Description

External callers should not need changes, or even be aware, when we replace the RESTBase "lists" endpoints with MediaWiki REST API endpoints. To help ensure this, create a testing plan to ensure compatibility.

Ideally, this will include tests that can be run against both the RESTBase endpoints and the new MediaWiki REST API endpoints, to ensure compatibility. This may mean an external testing suite, such as was used for AQS 2.0, is appropriate. But any approach that achieves the goal can be considered.

One complication compared to AQS 2.0 is that the Reading Lists functionality includes endpoints that modify data.

See the associated Miro board for endpoint map.
See parent task T336693: Re-implement reading lists REST interface outside RESTbase for context and details.

We have three levels of testing planned:

  1. unit tests: these are implemented within the extension. These exercise the contract of each individual handler and trait that are part of the new REST API implementation. Because these run on every git (gerrit) push, they are useful for discovering regressions vs expected behavior. But because they do not actively compare behavior to the existing production endpoints, they are susceptible to missing subtle incompatibilities. And because they focus on the internals, they could miss issues with the behavior of the system as a whole.
  2. mocha tests: these are implemented within the extension. They exercise the new REST endpoints from an outside-in perspective and test the behavior of the system as a whole. This is very valuable. However, they are less effective than unit tests at exercising the internals of the handlers in detail, and they also do not compare the behavior of the old and new systems.
  3. compatibility script: this is an external Python script designed to be executed against both the old and the new endpoints. As such, it is less effective than either the unit tests or the mocha tests at testing internals, and cannot be used to test any additions made in the new REST API. However, it is capable of comparing the behavior of the old and new system, helping ensure callers will not be surprised by any unexpected changes.

Event Timeline

Change 972408 had a related patch set uploaded (by Fgoodwin; author: Fgoodwin):

[mediawiki/extensions/ReadingLists@master] WIP: REST test suite

https://gerrit.wikimedia.org/r/972408

Change 976266 had a related patch set uploaded (by Fgoodwin; author: Fgoodwin):

[mediawiki/extensions/ReadingLists@master] Work in progress: Rest Get Lists

https://gerrit.wikimedia.org/r/976266

Change 980057 had a related patch set uploaded (by Fgoodwin; author: Fgoodwin):

[mediawiki/extensions/ReadingLists@master] Broken: Handler Tests

https://gerrit.wikimedia.org/r/980057

BPirkle triaged this task as Medium priority.
BPirkle added a subscriber: FGoodwin.

Change 1008069 had a related patch set uploaded (by BPirkle; author: BPirkle):

[mediawiki/extensions/ReadingLists@master] Adjust paths for REST setup and teardown handlers to match production

https://gerrit.wikimedia.org/r/1008069

Change 1008069 merged by jenkins-bot:

[mediawiki/extensions/ReadingLists@master] Adjust paths for REST setup and teardown handlers to match production

https://gerrit.wikimedia.org/r/1008069