Orphaned linkrecommendation search index entries on beta cswiki
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Tgr
	Apr 20 2021, 12:38 PM

Description

The vast majority of beta cswiki link recommendations seem broken: present in the search index but not the database.

Example items from @RHo:

Details

	Subject	Repo	Branch	Lines +/-
	Fix link recommendation SearchDataForIndex handler	mediawiki/extensions/GrowthExperiments	REL1_36	+143 -11
	Fix link recommendation SearchDataForIndex handler	mediawiki/extensions/GrowthExperiments	master	+143 -11

Customize query in gerrit

Event Timeline

Tgr created this task.Apr 20 2021, 12:38 PM

Restricted Application added a project: Growth-Team. · View Herald TranscriptApr 20 2021, 12:38 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Checking the first 10 link recommendations, 2 work, 8 show the error dialog. Of the 8 one has been edited (so maybe it's a case of not updating the search index), the other 7 haven't so there isn't any obvious reason for the DB row to not be present.

IIRC the maintenance/fixLinkRecommendationData.php script had some consistency issues in its first incarnation (as did refreshLinkRecommendations.php), maybe we should remove all DB and search entries and start over?

(Beta has 447 tasks right now, it had 421 when I set up link recommendations there and there were no such issues then, so this is not a case of new entries added in some incorrect way; functional recommendations broke somehow.)

kostajh updated the task description. (Show Details)Apr 20 2021, 1:07 PM

kostajh added a subscriber: RHo.

In T280667#7020012, @kostajh wrote:

IIRC the maintenance/fixLinkRecommendationData.php script had some consistency issues in its first incarnation (as did refreshLinkRecommendations.php), maybe we should remove all DB and search entries and start over?

I think both of those worked correctly at the time when I ran them on beta; and in any case we did not have a noticeable amount of broken tasks then.

MariaDB [cswiki]> select count(*) from growthexperiments_link_recommendations;
+----------+
| count(*) |
+----------+
|      146 |
+----------+
1 row in set (0.00 sec)

@Tgr sounds like you're looking into it this week; feel free to bump the priority if you think it should be done ahead of other things. (Unexpected disappearances of recommendations from the DB does sound like it's worth investigating sooner rather than later.)

Change 681405 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] [WIP] Fix link recommendation SearchDataForIndex handler

https://gerrit.wikimedia.org/r/681405

gerritbot added a project: Patch-For-Review.Apr 20 2021, 2:32 PM

Yeah, the logic for deleting recommendations is badly broken.

Tgr moved this task from Incoming to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.Apr 20 2021, 2:36 PM

Tgr moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Apr 20 2021, 10:00 PM

Change 681405 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Fix link recommendation SearchDataForIndex handler

https://gerrit.wikimedia.org/r/681405

ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.3; 2021-04-27).Apr 22 2021, 6:00 PM

Maintenance_bot removed a project: Patch-For-Review.Apr 22 2021, 6:11 PM

In T280667#7028000, @gerritbot wrote:

Change 681405 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Fix link recommendation SearchDataForIndex handler

https://gerrit.wikimedia.org/r/681405

@Tgr with that patch merged, what's the next step? Re-run refreshLinkRecommendations and/or the fixLinkRecommendationData script?

kostajh moved this task from April 19 - April 23 to April 26 - April 30 on the Add-Link board.Apr 26 2021, 9:36 AM

Yes, run fixLinkRecommendationData to remove the affected pages from the search index (since refreshLinkRecommendations relies on the index for finding candidates with no recommendation) and then run refreshLinkRecommendations to recreate the DB entry and re-add to the index.

...although with the cronjobs on beta now fixed, refreshLinkRecommendations is running automatically anyway, so running the other one should suffice.

OK, I've just run mwscript extensions/GrowthExperiments/maintenance/fixLinkRecommendationData.php --wiki=cswiki --search-index.

Seems OK now.

Done; spot-tested on cswiki and 10 out of 10 tasks work now.

I don't think this is directly QA-able so I'll just close it.

(Sorry, looks like we've been working in parallel. I forgot to refresh the task before commenting.)

Change 778628 had a related patch set uploaded (by Umherirrender; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@REL1_36] Fix link recommendation SearchDataForIndex handler

https://gerrit.wikimedia.org/r/778628

gerritbot added a project: Patch-For-Review.Apr 9 2022, 4:04 PM

Change 778628 merged by Umherirrender: