Page MenuHomePhabricator

Ensure growthimagesuggestiondata API works on all Wikipedias
Closed, ResolvedPublic

Description

The growthimagesuggestiondata API needs the image-recommendation and section-image-recommendation task types defined in the wiki's community configuration, otherwise using it will result in T337330. The Android app will start using that API on all Wikipedias in August. We need to update the community configuration (this should have no user-visible effects as long as $wgGENewcomerTasksImageRecommendationsEnabled / $wgGENewcomerTasksSectionImageRecommendationsEnabled are false), or maybe add some default task type configuration at the code level.

Event Timeline

Urbanecm_WMF triaged this task as Medium priority.
Urbanecm_WMF subscribed.

It seems this should only need the community configuration changes, which is trivial.

It seems this should only need the community configuration changes, which is trivial.

Except newly created Wikipedias, of course, which we still need to cover in some way in T294366: Think about Growth features being used by new wikis, deployed from the Incubator, so strictly speaking, it will not work on all Wikipedias immediately after completing this task.

Here is a list of Wikipedias that currently have no Growth features:

urbanecm@Martins-MacBook-Pro mediawiki-config % php multiversion/bin/expanddblist 'wikipedia - closed - private - growthexperiments'
anpwiki
blkwiki
fatwiki
gpewiki
gucwiki
gurwiki
guwwiki
kcgwiki
pcmwiki
urbanecm@Martins-MacBook-Pro mediawiki-config %

The community configuration changes are now done and the API now should work everywhere. I think we should also add some default task configuration on the code level, but that doesn't necessarily need to happen now (or as a Top Product Priority).

I think we should also add some default task configuration on the code level, but that doesn't necessarily need to happen now (or as a Top Product Priority).

Split as T341449: Provide default task type configuration for communities.

The API should be now be available on all Wikipedias. Example URL (for enwiki, but it should work everywhere): section-image-recommendation.

@Urbanecm_WMF @Tgr Thanks for this! One slightly related note:
The newly-added AddImageFeedbackHandler (T322309) doesn't seem to work on wikis where the task type is not enabled. Will we not be able to use that API in this case, and/or should we use growthinvalidateimagerecommendation instead?

(actually the growthinvalidateimagerecommendation also seems to fail with Caught exception of type Wikimedia\Assert\ParameterTypeException if the task is not enabled)

@Urbanecm_WMF @Tgr Thanks for this! One slightly related note:
The newly-added AddImageFeedbackHandler (T322309) doesn't seem to work on wikis where the task type is not enabled. Will we not be able to use that API in this case, and/or should we use growthinvalidateimagerecommendation instead?

(actually the growthinvalidateimagerecommendation also seems to fail with Caught exception of type Wikimedia\Assert\ParameterTypeException if the task is not enabled)

Good question. The invalidation API verifies the page/image pair was actually suggested to the user in Special:Homepage. Unfortunately, this validation check makes it difficult for the API to be used by the Android app in its current form (even if it worked; the exception itself is tracked as T341813). This check is included to limit the possibility of abusers to drain the task pool by invalidating all articles that exist.

I'm not sure how would one rewrite the check to be working for third party users, apart from introducing some sort of credentials that would allow MediaWiki to verify the request came from a trusted source. @Tgr @Sgs, do you have any thoughts on this please?

I'm not sure how would one rewrite the check to be working for third party users, apart from introducing some sort of credentials that would allow MediaWiki to verify the request came from a trusted source. @Tgr @Sgs, do you have any thoughts on this please?

Copying what I said in Slack: I think the validation check would not be that effective against someone trying to drain the task pool out by mass-invalidating, assuming the attacker knows how the APIs work internally, as it is trivial to find out which articles are in the task pool.

Consequently, I think we should switch to a different approach instead. Rate limiting should help with slowing those attempts down, although not prevent them. We allow users to complete up to 25 tasks a day -- there shouldn't be any reason to invalidate more than ~50 tasks a day (the additional 25 are for wikis that decide to increase the number of daily tasks).

@Tgr @Sgs Do you have any thoughts on this approach?

Change 940513 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] api: Expose disabled task types in ApiInvalidateImageRecommendation

https://gerrit.wikimedia.org/r/940513

Change 940516 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] api: Expose disabled task types in AddImageFeedbackHandler

https://gerrit.wikimedia.org/r/940516

AddImageFeedbackHandler also invalidates, and I don't think we restrict it in any way, so the attack opportunity already exists. Maybe just remove the check and get back to it if it becomes a problem?

AddImageFeedbackHandler also invalidates, and I don't think we restrict it in any way, so the attack opportunity already exists. Maybe just remove the check and get back to it if it becomes a problem?

Yeah, that sounds like a reasonable option to me. Not sure if we want to rate limit it now (since we're already touching the endpoint now), but considering the feedback handler doesn't do that, it probably makes sense just removing the problematic check and returning to this if we ever see a problem coming from here.

Change 940516 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] api: Expose disabled task types in AddImageFeedbackHandler

https://gerrit.wikimedia.org/r/940516

Change 940513 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] api: Expose disabled task types in ApiInvalidateImageRecommendation

https://gerrit.wikimedia.org/r/940513

@Urbanecm_WMF @KStoller-WMF since you made your recent changes, android users are accepting image recommendations at dewiki, even though our community never enabled this feature (compare e.g. de:Spezial:NewcomerTasksInfo with fr:Spécial:NewcomerTasksInfo). I tried to fix this at de:Spezial:EditGrowthConfig, but there was no option, so I tried to disable it manually via de:MediaWiki:NewcomerTasks.json. So far this doesn't seem to have any affect, currently there are still image recommendations being displayed using the android app. Is there something else that needs to be changed?

@Johannnes89, sorry for the confusion! de:Spezial:EditGrowthConfig relates only the website and won't impact the Android app. I'll connect with the Android app team / @JTannerWMF to determine if they can assist.

Thanks for pinging me @KStoller-WMF , and thanks for reaching out about this @Johannnes89.

My name is Jaz and I am the Product Manager on the apps. The image recommendations implementation, like the image recommendation on Structured Data is implemented a bit differently than image recommendations through the Growth experiments, which is why the process for switching off the feature isn't universally the same. The current community config for the feature is to turn it off so that newcomers (new editors), are not able to access the feature through Growth experiments (largely on Web). However, experienced users using the structured data implementation of it and experienced users on Android can still use the feature to add images.

Only logged in editors with 50+ unreverted edits in the article main namespace may use the image recommendations feature on Android. On Structured data, only logged in editors with 500+ edits will receive a notification prompting them to add a suggested image to an article. So far, our data tells us there significantly less reverts through the Android and Structured Data implementation, I assume due to it being more experienced users using the tool. However I can imagine it being confusing seeing it come through a recent edits feed when the Growth implementation did not have it enabled.

To reduce confusion, I am working with @KStoller-WMF to update the community config to distinguish how to shut off each implementation type or change the perimeters. To address your specific concerns on our end (for the apps) we can also do the following:

  • Increase the threshold of edits someone on German Wikipedia must have to access the feature if 50+ feels too low
  • Share with you the revert rate for German Wikipedia specifically so that the community can discuss what the best threshold is
  • Proceed with disabling this suggested edit type on Android completely, which would require submitting a phab task to our team (android-app) to reenable it should folks in your language community like it reintroduce in the future. I can also let you know if we start receiving complaints about it not being available via the apps support email or the Google Play Store