Page MenuHomePhabricator

Spike - Title normalization is not applied when saving pages to the reading list via the API
Closed, ResolvedPublic3 Estimated Story Points

Description

The reading list API stores page titles as opaque strings with no normalization. The DB unique constraint on (rle_rl_id, rle_rlp_id, rle_title) only prevents exact-string duplicates.

Different clients submit titles with different formats:

  • Desktop / mobile web saves pages with underscores (wgPageName), via the action API
  • Android saves with underscores (prefixedText) and handles normalizing on the client
  • iOS saves page titles with spaces to the reading list.

This can result in having duplicate entries "United Arab Emirates" and "United_Arab_Emirates" both in the reading list for the @local wiki. This makes it more difficult to check if a page is bookmarked (to show the bookmark button as "Save" or "Unsave"), among other issues.

As well, when reading list entries are retrieved from the API by iOS, the API returns the page title with however it was stored in the database, so it might have underscores or might not. The Android app appears to account for this and formats the page titles for display without underscores, but the iOS app does not do this.

Proposed fix

Add server-side normalization for ReadingLists titles so the backend no longer depends on clients sending the page titles in a particular format.

  • Convert spaces to underscores in page titles when storing reading list entries in the database.
  • When performing a look up in the database, convert the page title to have underscores as input to lookup DB queries.
  • In API responses, convert page titles to have spaces, which is more display friendly and common approach for MediaWiki APIs. This would allow us to implement these changes, while not impacting mobile apps.
  • We should implement and run a maintenance script to cleanup of existing mixed-format rows and then remove workaround where we check the database or account for page titles stored in both formats.

Possible follow-up:

  • for @local entries only, consider stronger normalization via Title::newFromText()->getPrefixedDBkey()
NOTE: On the web, we can work around this in the short term by handling this via the bloom filter and then when we need to do a database lookup a reading list entry, we check for both underscore (normalized) and non-normalized variations.

For mobile apps, we should make sure to return pages in the api with spaces so that pages saved on web will appear in the reading lists on iOS and Android.

Details

Event Timeline

We can do a shorter-term workaround via the bloom filter approach being implemented in T417404, but it is still very important to implement a more proper fix for this.

aude renamed this task from Title normalization is not applied when saving pages to the reading list via the API to Spike - Title normalization is not applied when saving pages to the reading list via the API.Mar 18 2026, 4:52 PM
aude set the point value for this task to 3.
aude moved this task from Needs refinement to Ready for sprint on the Reader Experience Team board.
aude updated the task description. (Show Details)

I put together a more detailed proposal as an ADR document (in proposed status):

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReadingLists/+/1260030

Change #1260030 had a related patch set uploaded (by Aude; author: Aude):

[mediawiki/extensions/ReadingLists@master] Add proposed title normalization ADR doc

https://gerrit.wikimedia.org/r/1260030

HFan-WMF lowered the priority of this task from High to Medium.Mar 25 2026, 6:15 PM
HFan-WMF subscribed.

Setting to medium in the context of the sprint.

If the proposed approach is good, then tasks:

  • Add title normalization on write: ReadingListRepository::addListEntry()
  • Add title normalization on lookup: ReadingListRepository::getListsByPage() and ListsPagesHandler
  • Add formatting for titles on API reads: ReadingListsHandlerTrait.php and ApiTrait.php

Optional: support @local project normalization with Title::newFromText() and Title::getPrefixedDBkey() when storing reading list entry page titles in the DB.

Then:

  • Create maintenance script, similar to fixListSize.php, to cleanup duplicate reading list entries, soft-deleting the entries with spaces if there exists duplicates (with and without spaces).

Then:

  • Remove workaround in BookmarkEntryLookupService.php and ReadingListRepository::getEquivalentPageTitles

Change #1260030 merged by jenkins-bot:

[mediawiki/extensions/ReadingLists@master] Add proposed title normalization ADR doc

https://gerrit.wikimedia.org/r/1260030

Jdlrobson-WMF claimed this task.