Exception: All recommendations were filtered for page: البيئة_في_لبنان
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	Sgs
	Dec 13 2021, 11:02 PM

Description

Error

mwversion: 1.38.0-wmf.12
reqId: da9b00d4-4a85-47ff-9856-9c66fceb476e
Find reqId in Logstash

normalized_message

[{reqId}] {exception_url}   Exception: All recommendations were filtered for page: البيئة_في_لبنان

exception.trace

from /srv/mediawiki/php-1.38.0-wmf.12/extensions/GrowthExperiments/includes/Util.php(181)
#0 /srv/mediawiki/php-1.38.0-wmf.12/extensions/GrowthExperiments/includes/HomepageHooks.php(371): GrowthExperiments\Util::logStatus(StatusValue)
#1 /srv/mediawiki/php-1.38.0-wmf.12/includes/HookContainer/HookContainer.php(160): GrowthExperiments\HomepageHooks->onBeforePageDisplay(OutputPage, SkinMinerva)
#2 /srv/mediawiki/php-1.38.0-wmf.12/includes/HookContainer/HookRunner.php(935): MediaWiki\HookContainer\HookContainer->run(string, array, array)
#3 /srv/mediawiki/php-1.38.0-wmf.12/includes/OutputPage.php(2686): MediaWiki\HookContainer\HookRunner->onBeforePageDisplay(OutputPage, SkinMinerva)
#4 /srv/mediawiki/php-1.38.0-wmf.12/includes/MediaWiki.php(909): OutputPage->output(boolean)
#5 /srv/mediawiki/php-1.38.0-wmf.12/includes/MediaWiki.php(922): MediaWiki::{closure}()
#6 /srv/mediawiki/php-1.38.0-wmf.12/includes/MediaWiki.php(554): MediaWiki->main()
#7 /srv/mediawiki/php-1.38.0-wmf.12/index.php(53): MediaWiki->run()
#8 /srv/mediawiki/php-1.38.0-wmf.12/index.php(46): wfIndexMain()
#9 /srv/mediawiki/w/index.php(3): require(string)
#10 {main}

Impact

Notes

Details

Request URL: https://ar.wikipedia.org/w/index.php?title=*&getasktype=*&gesuggestededit=*&veaction=*&section=*&geclickid=*&genewcomertasktoken=*

	Subject	Repo	Branch	Lines +/-
	Add an image: log reason for filtering all files in a suggestion	mediawiki/extensions/GrowthExperiments	master	+77 -15
	Add an image: track all suggestions filtered errors as non fatal	mediawiki/extensions/GrowthExperiments	master	+32 -1

Customize query in gerrit

Related Objects

Mentioned In: T297406: Exception: No recommendation found for page: *
Mentioned Here: T293711: [EPIC] "Add an image" Iteration 2
T300851: [EPIC] Growth: "add a link" structured task 2.0
T296333: Add an image: exclude non-images

Event Timeline

Sgs created this task.Dec 13 2021, 11:02 PM

Restricted Application added subscribers: Masumrezarock100, Aklapper. · View Herald TranscriptDec 13 2021, 11:02 PM

Probably related to T296333: Add an image: exclude non-images. Should we keep track of it in case the number of events spikes or add it to the ignore filters? @kostajh @Tgr

Might be worth checking the reason the images were excluded the first few times we see this, just to make sure filtering is behaving correctly, and to get a better idea of what things tend to get filtered (as in the long term we'd want that filtering to happen upstream).

Sgs claimed this task.Dec 14 2021, 10:00 AM

Change 747149 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] Add an image: track all suggestions filtered errors as non fatal

https://gerrit.wikimedia.org/r/747149

gerritbot added a project: Patch-For-Review.Dec 14 2021, 4:03 PM

Sgs moved this task from Inbox to Sprint 0 (Growth Team) on the Growth-Team board.Dec 14 2021, 4:04 PM

Sgs edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.

Sgs moved this task from Incoming to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.

Sgs moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Dec 14 2021, 6:19 PM

kostajh mentioned this in T297406: Exception: No recommendation found for page: *.Dec 15 2021, 8:03 AM

kostajh moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.

Change 747149 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Add an image: track all suggestions filtered errors as non fatal

https://gerrit.wikimedia.org/r/747149

ReleaseTaggerBot added a project: MW-1.38-notes (1.38.0-wmf.16; 2022-01-03).Dec 15 2021, 9:00 AM

Maintenance_bot removed a project: Patch-For-Review.Dec 15 2021, 9:10 AM

Etonkovidova moved this task from QA to Test in Production | Watching on the Growth-Team (Sprint 0 (Growth Team)) board.Dec 21 2021, 11:58 PM

Krinkle moved this task from Untriaged to Dec 2021 on the Wikimedia-production-error board.Jan 11 2022, 11:53 PM

The error showed again in Logstash. I will check what's the reason for filtering again. It is suspicious we're only got results filtered in arwiki so far.

The 2 errors we have seen for this are for fair reasons. البيئة_في_لبنان has only 1 suggestion which is a video file. قانون_الاستثمار_الأجنبي_لجمهورية_الصين_الشعبية has 1 suggestion which is a PDF file.

I will keep monitoring this errors since they have only showed fo arwiki so far. Would like to see an equal distribution on other wikis, otherwise it might be telling us the suggestion algorithm has particular flaws with arabian language pages.

Sgs moved this task from Test in Production | Watching to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.Jan 19 2022, 11:57 AM

Sgs moved this task from In Progress to Test in Production | Watching on the Growth-Team (Sprint 0 (Growth Team)) board.Jan 20 2022, 9:59 AM

In T297656#7631992, @Sgs wrote:

The error showed again in Logstash. I will check what's the reason for filtering again. It is suspicious we're only got results filtered in arwiki so far.

The 2 errors we have seen for this are for fair reasons. البيئة_في_لبنان has only 1 suggestion which is a video file. قانون_الاستثمار_الأجنبي_لجمهورية_الصين_الشعبية has 1 suggestion which is a PDF file.

I will keep monitoring this errors since they have only showed fo arwiki so far. Would like to see an equal distribution on other wikis, otherwise it might be telling us the suggestion algorithm has particular flaws with arabian language pages.

I checked logstash for Dec12, 2021-Jan 21, 2022 (https://logstash.wikimedia.org/goto/fd76f3970e574cf958bb8fadde22b7c1) the count is 46). 39 errors are reported on arwiki and 7 errors are on bnwiki.

Re-opening this since the error count is adding up, suspiciously for arwiki and bnwiki. Thank you for pointing this @Etonkovidova

In T297656#7674171, @Sgs wrote:

Re-opening this since the error count is adding up, suspiciously for arwiki and bnwiki. Thank you for pointing this @Etonkovidova

FWIW the count doesn't seem very high to me. The highest daily number of errors is 7 on February 1; in total there are 56 instances since Dec 12. It does seem suspicious that it is mostly on arwiki; but I'd suggest we make a new task to investigate that as part of T300851: [EPIC] Growth: "add a link" structured task 2.0, and maybe it is something for @MGerlach or Research to focus on.

In T297656#7674470, @kostajh wrote:

I'd suggest we make a new task to investigate that as part of T300851: [EPIC] Growth: "add a link" structured task 2.0, and maybe it is something for @MGerlach or Research to focus on.

Did you mean T293711: [EPIC] "Add an image" Iteration 2?

I think the immediate issue here is that we don't log the reason for the filtering (a new status gets constructed and the old one discarded) so we should probably fix that. Beyond that, it's probably just a matter of bringing the algorithm more in sync with the filtering (e.g. not recommending videos).

Sgs changed the task status from Open to In Progress.Feb 4 2022, 4:32 PM

Sgs moved this task from Ready for Development to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.

In T297656#7676602, @Tgr wrote:

In T297656#7674470, @kostajh wrote:

I'd suggest we make a new task to investigate that as part of T300851: [EPIC] Growth: "add a link" structured task 2.0, and maybe it is something for @MGerlach or Research to focus on.

Did you mean T293711: [EPIC] "Add an image" Iteration 2?

I think the immediate issue here is that we don't log the reason for the filtering (a new status gets constructed and the old one discarded) so we should probably fix that. Beyond that, it's probably just a matter of bringing the algorithm more in sync with the filtering (e.g. not recommending videos).

Oops, indeed. I thought it was about Add-Link. Sorry!

Change 760526 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] Add an image: log reason for filtering all files in a suggestion

https://gerrit.wikimedia.org/r/760526

gerritbot added a project: Patch-For-Review.Feb 7 2022, 11:54 AM

In T297656#7676602, @Tgr wrote:

I think the immediate issue here is that we don't log the reason for the filtering (a new status gets constructed and the old one discarded) so we should probably fix that. Beyond that, it's probably just a matter of bringing the algorithm more in sync with the filtering (e.g. not recommending videos).

The patch will log the reasons for filtering. Another common case I'm seeing which is worth to take in account for syncing with the algorithm is PDF files being offered as images.

Sgs moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.Feb 8 2022, 12:19 PM

We should probably also make these warnings (Growth channel messages) instead of production errors, as there is not too much we can do about them.

Change 760526 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Add an image: log reason for filtering all files in a suggestion

https://gerrit.wikimedia.org/r/760526

ReleaseTaggerBot edited projects, added MW-1.38-notes (1.38.0-wmf.22; 2022-02-14); removed MW-1.38-notes (1.38.0-wmf.16; 2022-01-03).Feb 9 2022, 10:00 PM

Maintenance_bot removed a project: Patch-For-Review.Feb 9 2022, 10:11 PM

In T297656#7698644, @Tgr wrote:

We should probably also make these warnings (Growth channel messages) instead of production errors, as there is not too much we can do about them.

These are already falling into GrowthExperiments channel. Even if we can't do much about them I think they are interesting to monitor what's the recommendation offering and we don't like. That said maybe this is not best way to do it and we should just not log these.

I'm wondering if it is possible and if it would make sense to adapt the image-suggestions API to accept some parameters like mime=bitmap&minwidth=200.

Moving this task to QA since that would require more analysis and its own task.

Sgs changed the task status from In Progress to Open.Feb 10 2022, 10:45 AM

Sgs moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.

Etonkovidova moved this task from QA to Test in Production | Watching on the Growth-Team (Sprint 0 (Growth Team)) board.Feb 10 2022, 7:00 PM

In T297656#7700177, @Sgs wrote:

In T297656#7698644, @Tgr wrote:

We should probably also make these warnings (Growth channel messages) instead of production errors, as there is not too much we can do about them.

These are already falling into GrowthExperiments channel. Even if we can't do much about them I think they are interesting to monitor what's the recommendation offering and we don't like. That said maybe this is not best way to do it and we should just not log these.

I'm wondering if it is possible and if it would make sense to adapt the image-suggestions API to accept some parameters like mime=bitmap&minwidth=200.

Moving this task to QA since that would require more analysis and its own task.

There are no error registered for wmf.22 - Jan 21-Feb 17/2022- https://logstash.wikimedia.org/goto/07738a4f3f02157167985f3bbef89ee0 . I agree that it'd be interesting to monitor for the issue. Closing this particular task as Resolved. If the error would appear again, a new task might be created if needed.

Exception: All recommendations were filtered for page: البيئة_في_لبنانClosed, ResolvedPublicPRODUCTION ERRORActions