Our next A/B test is the Minerva TOC feature. This feature has two variants (?mpo=mobile-toc-abc:treatment1 and ?mpo=mobile-toc-abc:treatment2), and targets logged-out users of the mobile site (Minerva skin only).
Per T416503, we will only conduct this experiment on Parsoid-rendered views. This means that both control and experiment groups will be opted in to Parsoid. We won't test any experiment features against legacy-parser-rendered content.
Before we launch the Minerva TOC experiment, let's check the following things:
Analytics
- Confirm that the experiment. has been created in Test Kitchen and has the correct start and end dates - https://test-kitchen.wikimedia.org/experiment/mobile-toc-abc - start date: Feb 24 2026 - end date: Mar 24 2026
- Confirm that all appropriate events are being submitted according to the instrumentation spec (this should be done locally in dev environment) – this should be tested for both feature variants as well as the control group
Content test
- Both versions of the feature have been tested in all relevant language wikis
- Arabic Wikipedia
- Chinese Wikipedia
- French Wikipedia
- Indonesian Wikipedia
- Vietnamese Wikipedia
- English Wikipedia
- We've tested the feature (in both variants) against a known list of pages that have "challenging" content (complex layouts, mathematical notation, overflow, tables, complex templates, etc). Here are some to check (list in progress):
- https://en.wikipedia.org/wiki/Bernoulli_number
- https://en.wikipedia.org/wiki/Swan_maiden
- https://en.wikipedia.org/wiki/International_Space_Station
- https://en.wikipedia.org/wiki/Apollo
- https://en.wikipedia.org/wiki/The_Beach_Boys
- https://en.wikipedia.org/wiki/Microsoft_Windows_version_history
- https://en.wikipedia.org/wiki/Korean_War
- https://en.wikipedia.org/wiki/Congress_of_the_Philippines
- https://en.wikipedia.org/wiki/List_of_Book_of_Mormon_people
⛔ - empty ToC- fixed - https://en.wikipedia.org/wiki/List_of_Dungeons_%26_Dragons_3rd_edition_monsters
- https://en.wikipedia.org/wiki/List_of_Dungeons_%26_Dragons_3.5_edition_monsters
Accessibility test
- The feature (both variants) has been tested in VoiceOver on iOS and doesn't exhibit any serious new breakages
- The feature (both variants) has been tested for keyboard navigation UX and doesn't exhibit any serious new breakages
Patches to backport
[] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReaderExperiments/+/1240796 - abandoned
- https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReaderExperiments/+/1240659
- https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReaderExperiments/+/1240660
- https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReaderExperiments/+/1241034
[] https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReaderExperiments/+/1242805 - not included