Page MenuHomePhabricator

Lists for some users being incorrectly indexed as public lists
Closed, ResolvedPublicBUG REPORT

Description

More lists than expected are being indexed, and some lists expected to be indexed are not being indexed:

toolhub_admin@m5-master.eqiad.wmnet(toolhub)> select id from lists_toollist where published=1;
+----+
| id |
+----+
|  1 |
|  5 |
|  6 |
| 22 |
+----+
4 rows in set (0.001 sec)
$ curl -s -XPOST -H "Content-Type: application/json" -k 'https://search.svc.eqiad.wmnet:9243/toolhub_lists/_search?pretty' --data '{"_source":["id"]}' | jq '.hits.hits[]._source.id'
1
6
22
40
41
42
43
44
45
46

It looks like at least part of the problem here is that newly created lists (40-46) are indexed no matter if they have their published flag set or not. This seems similar to T303892: List not removed from search index when un-published, but it is not exactly the same thing.

Original report by @Quiddity:
The listing at https://toolhub.wikimedia.org/published-lists shows many empty lists. (4-10)

If I create my own list, and click the "link" icon, I see this error

Oops! An error occurred in detail: Not found.

image.png (668×1 px, 54 KB)

Event Timeline

bd808 renamed this task from New Published lists (Favorite listings) are empty and broken to Lists for some users being incorrectly indexed as public lists.Apr 15 2022, 8:11 PM
bd808 triaged this task as High priority.
bd808 updated the task description. (Show Details)
bd808 moved this task from Backlog to Groomed/Ready on the Toolhub board.

After doing some local testing, I think that T303892: List not removed from search index when un-published actually does fix this bug as well if only by accident. Specifically the bug in the initial implementation described at T303892#7798046 is proof of the not instance.published guard working on initial save. If I comment out that block I can recreate the prod bug in my local environment.

I think all we actually need to do to fix this production issue is to make a new release and re-create the lists index. The patches for T303892 should prevent additional unexpected indexing and the re-creation will clear out the improperly indexed lists.

bd808 moved this task from Groomed/Ready to Review on the Toolhub board.

Confirmed that following the recent deployment of rWTHU18d94da679c5: ui: Fix reactivity issues when deleting lists through the ToolMenu attaching new user does not publish an empty copy of their favorites list. The elasticsearch index rebuilds done as part of the production deployment also removed the incorrectly exposed lists from the index.