Page MenuHomePhabricator

Deploy "add a link" to 9th round of wikis
Closed, ResolvedPublic

Description

  • Training models
    • Hausa Wikipedia ha
    • Hakka Chinese Wikipedia hak
    • Hawaiian Wikipedia haw
    • Fiji Hindi Wikipedia hif
    • Croatian Wikipedia hr
    • Upper Sorbian Wikipedia hsb
    • Haitian Creole Wikipedia ht
    • ~Armenian Wikipedia hy~ see T308134#8539194
    • ~Western Armenian Wikipedia hyw~ see T308134#8548734
    • Interlingua Wikipedia ia
    • Interlingue Wikipedia ie
    • Igbo Wikipedia ig
    • Inupiaq Wikipedia ik
    • Iloko Wikipedia ilo
    • Ingush Wikipedia inh
    • Ido Wikipedia io
    • Icelandic Wikipedia is
    • Inuktitut Wikipedia iu
    • Jamaican Patois Wikipedia jam
    • Lojban Wikipedia jbo
    • Javanese Wikipedia jv
  • Models verification
  • Publish Datasets
  • Populate the excluded section titles
  • Deploy back-end
  • Check how the model works on the wikis
  • In Search, use hasrecommendation:link to find articles
  • Test them on https://api.wikimedia.org/service/linkrecommendation/apidocs/#/default/get_v1_linkrecommendations__project___domain___page_title_
  • Inform communities
  • Deploy front-end

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

20/21 models were trained successfully in the 9th round of wikis.

The Armenian Wikipedia (hywiki) pipeline did not complete successfully and is being investigated in T327371.

Model evaluation has been completed and below are the backtesting results:

Precision@0.5Recall@0.5
hawiki0.680.46
hakwiki0.920.75
hawwiki0.920.74
hifwiki0.930.73
hrwiki0.810.52
hsbwiki0.900.68
htwiki0.920.42
hywwiki0.630.16
iawiki0.940.73
iewiki0.980.93
igwiki0.720.49
ikwiki0.920.60
ilowiki0.910.65
inhwiki0.680.22
iowiki0.890.53
iswiki0.790.53
iuwiki0.960.72
jamwiki0.820.60
jbowiki0.700.29
jvwiki0.870.61

CCing @MGerlach, in case he would like to add comments on the backtesting evaluation.

The conclusion on the backtesting results is that most of the languages look fine besides:

  • hywwiki has a low precision (0.63) and recall (0.16).
  • igwiki (0.72), jbowiki (0.70), and both hawiki, inhwiki (0.68) have a precision that is slightly lower than the recommended one (0.75).

Talked to @MGerlach about these results and agreed: not to deploy hywwiki; but deploy igwiki, jbowiki, hawiki, and inhwiki since their precision is not too low and the recall is good.

kevinbazira added a subscriber: kostajh.

@kostajh, we published datasets for all 19/21 models that passed the evaluation in this round.

@kostajh, we published datasets for all 19/21 models that passed the evaluation in this round.

🎉 thanks!

Sgs moved this task from Triaged to Sprint 0 (Growth Team) on the Growth-Team board.
Sgs edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.
Sgs added a subscriber: kevinbazira.
Sgs changed the task status from Open to In Progress.Feb 24 2023, 11:58 AM

I ran this script for adding the link-recommendation task type and and populating the excluded sections:

PHAB=T308134
for WIKI in hawiki hakwiki hawwiki hifwiki hrwiki hsbwiki htwiki hywwiki iawiki iewiki igwiki ikwiki ilowiki inhwiki iowiki iswiki iuwiki jamwiki jbowiki jvwiki; do
    ORIGIN=`mwscript getConfiguration.php $WIKI --settings 'wgCanonicalServer' --format json | jq --raw-output '.wgCanonicalServer'`
    mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --create-only \
            --json \
            --summary "Growth features configuration boilerplate ([[phab:$PHAB]])" \
            link-recommendation \
            '{ "type": "link-recommendation", "group": "easy" }'
    jq "select(.wiki==\"$WIKI\" and .probability > 0.25) | .section" wiki_sections.jsonl \
        | jq --slurp --compact-output "unique" \
        | mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --json \
            --summary "machine-generated configuration for excluding sections from link recommendations ([[phab:$PHAB]]), feel free to improve" \
            link-recommendation.excludedSections \
            "`cat`"
    echo "$ORIGIN/wiki/MediaWiki:NewcomerTasks.json"
    echo "$ORIGIN/w/index.php?title=MediaWiki:NewcomerTasks.json&diff=next"
    echo "Press <Enter> to continue"
    read # give time for manual verification
done

I checked the configuration and it seemed to be correctly updated in all wikis. The only mentions worth are ikwiki and ilo which didn't get any excluded section.

Change 892365 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: Enable link recommendation for 9th round wikis

https://gerrit.wikimedia.org/r/892365

Change 892363 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: Enable backend of link recommendation for 7,8,9th round wikis

https://gerrit.wikimedia.org/r/892363

Change 892365 abandoned by Sergio Gimeno:

[operations/mediawiki-config@master] GrowthExperiments: Enable link recommendation for 9th round wikis

Reason:

squashed in I81293b799ec5afe62a19ac2d79e0434047cf1be2

https://gerrit.wikimedia.org/r/892365

Change 892363 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: Enable backend of link recommendation for 7, 8, 9th round wikis

https://gerrit.wikimedia.org/r/892363

Mentioned in SAL (#wikimedia-operations) [2023-03-15T20:13:23Z] <samtar@deploy2002> Started scap: Backport for [[gerrit:899673|GrowthExperiments: enable frontend of link recommendation for 6th round wikis (T304550)]], [[gerrit:892363|GrowthExperiments: Enable backend of link recommendation for 7, 8, 9th round wikis (T304551 T308133 T308134)]]

Mentioned in SAL (#wikimedia-operations) [2023-03-15T20:14:55Z] <samtar@deploy2002> sgimeno and samtar: Backport for [[gerrit:899673|GrowthExperiments: enable frontend of link recommendation for 6th round wikis (T304550)]], [[gerrit:892363|GrowthExperiments: Enable backend of link recommendation for 7, 8, 9th round wikis (T304551 T308133 T308134)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-03-15T20:23:36Z] <samtar@deploy2002> Finished scap: Backport for [[gerrit:899673|GrowthExperiments: enable frontend of link recommendation for 6th round wikis (T304550)]], [[gerrit:892363|GrowthExperiments: Enable backend of link recommendation for 7, 8, 9th round wikis (T304551 T308133 T308134)]] (duration: 10m 12s)

Sgs updated the task description. (Show Details)
Sgs added a subscriber: Sgs.

@Sgs, based on the fact that a majority of models aren't working for the previous round (T308133), I sampled this one, and all samples returned "There were no results matching the query.". Could you check again?

Change 905950 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: enable add link frontend and backend

https://gerrit.wikimedia.org/r/905950

@Sgs, based on the fact that a majority of models aren't working for the previous round (T308133), I sampled this one, and all samples returned "There were no results matching the query.". Could you check again?

I checked all wikis from this round, this is what I found:

hrwiki and hywiki were correctly enabled but they were missing in the wikis.txt file so not being populated. I've manually added them, cc @kevinbazira

From iewiki to jvwiki were not enabled in 892363, apologies for this, I must have missed them when squashing changes. They will be enabled today at 15h UTC+2.

We should check again all these wikis in a couple of days.

Change 905950 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: enable add link backend in wiki rounds (8,9th)

https://gerrit.wikimedia.org/r/905950

Mentioned in SAL (#wikimedia-operations) [2023-04-05T13:08:58Z] <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:905950|GrowthExperiments: enable add link backend in wiki rounds (8,9th) (T308133 T308134)]]

Mentioned in SAL (#wikimedia-operations) [2023-04-05T13:10:28Z] <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde and sgimeno: Backport for [[gerrit:905950|GrowthExperiments: enable add link backend in wiki rounds (8,9th) (T308133 T308134)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-04-05T13:16:58Z] <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:905950|GrowthExperiments: enable add link backend in wiki rounds (8,9th) (T308133 T308134)]] (duration: 08m 00s)

I tested the models, and everything works fine, except:

  • hr.wp returns "There were no results matching the query."
  • ie.wp returns "There were no results matching the query."
  • ig.wp returns "There were no results matching the query."
  • ik.wp returns "There were no results matching the query."
  • ilo.wp returns "There were no results matching the query."
  • inh.wp returns "There were no results matching the query."
  • io.wp returns "There were no results matching the query."
  • is.wp returns "There were no results matching the query."
  • iu.wp returns "There were no results matching the query."
  • is.wp returns "There were no results matching the query."
  • jam.wp returns "There were no results matching the query."
  • jbo.wp returns "There were no results matching the query."
  • jv.wp returns "There were no results matching the query."

I tested the models, and everything works fine, except:

  • hr.wp returns "There were no results matching the query."
  • ie.wp returns "There were no results matching the query."
  • ig.wp returns "There were no results matching the query."
  • ik.wp returns "There were no results matching the query."
  • ilo.wp returns "There were no results matching the query."
  • inh.wp returns "There were no results matching the query."
  • io.wp returns "There were no results matching the query."
  • is.wp returns "There were no results matching the query."
  • iu.wp returns "There were no results matching the query."
  • is.wp returns "There were no results matching the query."
  • jam.wp returns "There were no results matching the query."
  • jbo.wp returns "There were no results matching the query."
  • jv.wp returns "There were no results matching the query."

We're still investigating, I've added some more details in T308133#8774966 since it seems the same problem affecting gorwiki could also explain why there aren't results on this batch of wikis.

Any update?

All wikis have now results except jbowiki where link recommendations have been disabled on site. Also worth mentioning that ikwiki is only producing 5 results, so we might want to check that before enabling the frontend.

We can proceed informing communities for the 9th round (except jbo and ik) and add gorwiki from the 8th if you agree. I'll follow up with the issues in jbowiki and ikwiki.

Let's go then with gor + all round 9 (except jbo and ik). Can we deploy next Wednesday, May 17?

I'm adding it to Tech News.

Trizek-WMF set Due Date to May 17 2023, 4:00 PM.

Let's go then with gor + all round 9 (except jbo and ik). Can we deploy next Wednesday, May 17?

I'm adding it to Tech News.

Alright.

Change 920722 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: enable add link frontend in 9th round wikis

https://gerrit.wikimedia.org/r/920722

Change 920722 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: enable add link frontend in 9th round wikis

https://gerrit.wikimedia.org/r/920722

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:03:10Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:920722|GrowthExperiments: enable add link frontend in 9th round wikis (T308134)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:04:46Z] <urbanecm@deploy1002> sgimeno and urbanecm: Backport for [[gerrit:920722|GrowthExperiments: enable add link frontend in 9th round wikis (T308134)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet

Change 920784 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: amend wrong wiki prefix for jbowiki

https://gerrit.wikimedia.org/r/920784

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:15:17Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:920722|GrowthExperiments: enable add link frontend in 9th round wikis (T308134)]] (duration: 12m 06s)

Change 920784 merged by jenkins-bot:

[operations/mediawiki-config@master] GrowthExperiments: amend wrong wiki prefix for jbowiki

https://gerrit.wikimedia.org/r/920784

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:17:29Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:920784|GrowthExperiments: amend wrong wiki prefix for jbowiki (T308134)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (T317375)]], [[gerrit:920386|Enable zebra ab test in hewiki (T335972)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:19:04Z] <urbanecm@deploy1002> urbanecm and matmarex and ksarabia and sgimeno: Backport for [[gerrit:920784|GrowthExperiments: amend wrong wiki prefix for jbowiki (T308134)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (T317375)]], [[gerrit:920386|Enable zebra ab test in hewiki (T335972)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.

Mentioned in SAL (#wikimedia-operations) [2023-05-17T20:29:06Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:920784|GrowthExperiments: amend wrong wiki prefix for jbowiki (T308134)]], [[gerrit:920732|NewTopicOptOutActiveUsers: Skip bot users etc. (T317375)]], [[gerrit:920386|Enable zebra ab test in hewiki (T335972)]] (duration: 11m 36s)

Sgs changed the task status from In Progress to Open.May 18 2023, 11:35 AM
Sgs moved this task from In Progress to QA on the Growth-Team (Sprint 0 (Growth Team)) board.
Etonkovidova added a subscriber: Etonkovidova.

Checked Hawaiian Wikipedia haw, Icelandic Wikipedia is, Inuktitut Wikipedia iu, and Javanese Wikipedia jv - all works as expected.

I tested the models, and everything works fine, except:

  • hr.wp returns "There were no results matching the query."
  • ie.wp returns "There were no results matching the query."
  • ig.wp returns "There were no results matching the query."
  • ik.wp returns "There were no results matching the query."
  • ilo.wp returns "There were no results matching the query."
  • inh.wp returns "There were no results matching the query."
  • io.wp returns "There were no results matching the query."
  • is.wp returns "There were no results matching the query."
  • iu.wp returns "There were no results matching the query."
  • is.wp returns "There were no results matching the query."
  • jam.wp returns "There were no results matching the query."
  • jbo.wp returns "There were no results matching the query."
  • jv.wp returns "There were no results matching the query."

Checked those - only jbo.wp returns "There were no results matching the query." The Homepage is enabled on jbo.wp (there are options to enable it in Preferences) - no available Suggested edit tasks are displayed.

Screen Shot 2023-06-02 at 1.17.50 PM.png (1×1 px, 163 KB)

ikwiki - ikwiki Special:NewcomerTasksInfo does display 5 link-recommendation tasks available, but they are not present in SE module:

Screen Shot 2023-06-02 at 12.55.46 PM.png (1×1 px, 168 KB)

@Trizek-WMF , @Sgs - no follow-ups for jbo.wp and ikwiki ?

@Trizek-WMF , @Sgs - no follow-ups for jbo.wp and ikwiki ?

@Etonkovidova, we decided to exclude these two wikis earlier in this task: T308134#8841112. The task description should have been more explicit, sorry. These wikis will be treated later when all deployments are done.

As everything works as expected, I'm closing this task.