Page MenuHomePhabricator

Add an image: Enable on test wikis
Closed, ResolvedPublic

Description

beta

  • Enable image recommendations
  • Configure image suggestions API
  • Create or identify test articles
  • Add test articles to search index

testwiki

  • Enable image recommendations
  • Configure image suggestions API
  • Create or identify test articles
  • Add test articles to search index

Event Timeline

Change 720929 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[operations/mediawiki-config@master] [beta] GrowthExperiments: set image recommendation API URL

https://gerrit.wikimedia.org/r/720929

Change 720929 merged by jenkins-bot:

[operations/mediawiki-config@master] [beta] GrowthExperiments: set image recommendation API URL

https://gerrit.wikimedia.org/r/720929

We select a couple articles with image recommendations and use the import process from T274198#6972115. Testing save/reject will "use up" the test articles, we'll just have to live with that I think.

We select a couple articles with image recommendations and use the import process from T274198#6972115. Testing save/reject will "use up" the test articles, we'll just have to live with that I think.

Maybe we can use a hack of checking the GENewcomerTasksImageRecommendationsEnabled config setting; if it's disabled, then we wouldn't remove items from the search index. Although, we'd probably need a separate config setting like GENewcomerTasksImageRecommendationsFrontendEnabled, because switching GENewcomerTasksImageRecommendationsEnabled to false will hide the task type entirely.

kostajh triaged this task as Medium priority.Sep 24 2021, 12:38 PM
Add 500 test pages
WIKI_LANG=en
WIKI=testwiki
#WIKI=${WIKI_LANG}wiki

# fetch a list of pages which have recommendations
curl -s "https://image-suggestion-api.wmcloud.org/image-suggestions/v0/wikipedia/$WIKI_LANG/pages?limit=500&source=ima" -H 'accept: application/json' | jq --raw-output .pages[].page > pages-$WIKI.txt

# dump those pages
mediawiki-dump-downloader.py $WIKI_LANG.wikipedia.org pages-$WIKI.txt > dump-$WIKI.xml

# import the dump
mwscript importDump.php $WIKI dump-$WIKI.xml --no-local-users --username-prefix ${WIKI_LANG}wiki --report 50
mwscript rebuildrecentchanges.php $WIKI
mwscript initSiteStats.php $WIKI --update
mwscript extensions/CirrusSearch/maintenance/ForceSearchIndex.php $WIKI --namespace 0 --skipLinks --indexOnSkip
mwscript extensions/CirrusSearch/maintenance/ForceSearchIndex.php $WIKI --namespace 0 --skipParse

# update search index
mwscript extensions/GrowthExperiments/maintenance/importOresTopics.php $WIKI --apiUrl=https://$WIKI_LANG.wikipedia.org/w/api.php --wikiId=${WIKI_LANG}wiki --pageList pages-$WIKI.txt
cat pages-$WIKI.txt | xargs -d'\n' -n1 mwscript extensions/CirrusSearch/maintenance/UpdateWeightedTags.php $WIKI --tagType=recommendation.image --page

# validate
mwscript extensions/GrowthExperiments/maintenance/listTaskCounts.php $WIKI --topictype ores --tasktype image-recommendation --output json | jq '.taskTypeCounts["image-recommendation"]'

mediawiki-dump-downloader.py is a helper for downloading dumps and stitching them together, to get around the annoying 50 page per dump limit.

Change 731880 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[operations/mediawiki-config@master] [WIP] Enable GrowthExperiments Add Image feature on testwiki

https://gerrit.wikimedia.org/r/731880

The above is done for testwiki and beta en/ar/bn/cs/vi.

Change 732471 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/services/image-suggestion-api@master] Add /:wiki/:lang/pages/:title path

https://gerrit.wikimedia.org/r/732471

Change 732477 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] Add Image: Use title-based API endpoint

https://gerrit.wikimedia.org/r/732477

Change 731880 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable GrowthExperiments Add Image feature on testwiki

https://gerrit.wikimedia.org/r/731880

Still doesn't work on testwiki, needs some sort of proxy configuration.

Change 735093 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] Add Image: add HTTP proxy config

https://gerrit.wikimedia.org/r/735093

Change 735094 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[operations/mediawiki-config@master] Use url-downloader proxy for GrowthExperiments

https://gerrit.wikimedia.org/r/735094

Change 735093 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Add Image: add HTTP proxy config

https://gerrit.wikimedia.org/r/735093

Change 736314 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[operations/mediawiki-config@master] Use page id for GrowthExperiments image recommendations

https://gerrit.wikimedia.org/r/736314

Change 735094 merged by jenkins-bot:

[operations/mediawiki-config@master] Use url-downloader proxy for GrowthExperiments

https://gerrit.wikimedia.org/r/735094

Change 736314 merged by jenkins-bot:

[operations/mediawiki-config@master] Use page id for GrowthExperiments image recommendations

https://gerrit.wikimedia.org/r/736314

Mentioned in SAL (#wikimedia-operations) [2021-11-02T23:34:18Z] <tgr@deploy1002> Synchronized wmf-config/CommonSettings.php: Config: [[gerrit:735094|Use url-downloader proxy for GrowthExperiments (T290949)]] (duration: 01m 14s)

Change 736317 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[operations/mediawiki-config@master] Use title for GrowthExperiments image recommendations on testwiki

https://gerrit.wikimedia.org/r/736317

Change 736317 merged by jenkins-bot:

[operations/mediawiki-config@master] Use title for GrowthExperiments image recommendations on testwiki

https://gerrit.wikimedia.org/r/736317

Mentioned in SAL (#wikimedia-operations) [2021-11-02T23:45:50Z] <tgr@deploy1002> Synchronized wmf-config: Config: Use page id for GrowthExperiments image recommendations, except for testwiki ([[gerrit:736314|736314]] [[gerrit:736317|736317]] (T290949 T292154) (duration: 01m 03s)

Change 736518 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@wmf/1.38.0-wmf.6] Add Image: add HTTP proxy config

https://gerrit.wikimedia.org/r/736518

Change 736518 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.38.0-wmf.6] Add Image: add HTTP proxy config

https://gerrit.wikimedia.org/r/736518

Mentioned in SAL (#wikimedia-operations) [2021-11-04T00:01:53Z] <tgr@deploy1002> Synchronized php-1.38.0-wmf.6/extensions/GrowthExperiments: Backport: [[gerrit:736518|Add Image: add HTTP proxy config (T290949)]] [[gerrit:736519|Add Image: Harden API response parsing]] (duration: 01m 05s)

Etonkovidova subscribed.

Tested on testwiki and betalabs - works as expected.