Page MenuHomePhabricator

Deploy PageAssessments to Chinese Wikipedia
Closed, ResolvedPublic5 Estimated Story Points

Event Timeline

Restricted Application added subscribers: Ericliu1912, Stang, Aklapper. · View Herald Transcript

Change 876196 had a related patch set uploaded (by Stang; author: Stang):

[operations/mediawiki-config@master] zhwiki: Install PageAssessments

https://gerrit.wikimedia.org/r/876196

  • To deployer: Please run mwscript extensions/WikimediaMaintenance/createExtensionTables.php zhwiki pageassessments on mwmaint1002 before deploy to create tables.
  • Is it needed to add an alias for Special:PageAssessments on zhwiki? If so, what's the desired title?
MusikAnimal subscribed.

I'd be happy to help with this! I'm not a deployer, though.

Ping @Shizhao about this question:

  • Is it needed to add an alias for Special:PageAssessments on zhwiki? If so, what's the desired title?

I would guess an alias is preferred. Just let us know what it should be.

在T326387#8520349中,@MusikAnimal写道:

I'd be happy to help with this! I'm not a deployer, though.

Ping @Shizhao about this question:

  • Is it needed to add an alias for Special:PageAssessments on zhwiki? If so, what's the desired title?

I would guess an alias is preferred. Just let us know what it should be.

Hi, MusikAnimal! According to the discussions in our community, the alias should be "Special:页面评级".

Change 879832 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/PageAssessments@master] PageAssessments.i18n.alias.php: add alias for zh

https://gerrit.wikimedia.org/r/879832

Hi, MusikAnimal! According to the discussions in our community, the alias should be "Special:页面评级".

Got it! Patch created.

Next, let's review what needs to be done for the deployment. Do we have a template editor who will update Template:WPBannerMeta? Your WPBannerMeta template seems to be a bit different than enwiki's. I'm not sure exactly where to put the {{#assessment}} parser function. It looks like zhwiki does have a Template:Class mask, which I assume like enwiki is used to normalize the values passed in for assessments. It is important to use this normalization before passing values to {{#assessment}}, otherwise you end up with corrupt and/or inaccurate data.

I can try to figure out exactly what needs to be done, but perhaps you already know. As you might imagine, it's more difficult for me to review the template code as I don't understand Chinese.

在T326387#8524225中,@MusikAnimal写道:

Hi, MusikAnimal! According to the discussions in our community, the alias should be "Special:页面评级".

Got it! Patch created.

Next, let's review what needs to be done for the deployment. Do we have a template editor who will update Template:WPBannerMeta? Your WPBannerMeta template seems to be a bit different than enwiki's. I'm not sure exactly where to put the {{#assessment}} parser function. It looks like zhwiki does have a Template:Class mask, which I assume like enwiki is used to normalize the values passed in for assessments. It is important to use this normalization before passing values to {{#assessment}}, otherwise you end up with corrupt and/or inaccurate data.

I can try to figure out exactly what needs to be done, but perhaps you already know. As you might imagine, it's more difficult for me to review the template code as I don't understand Chinese.

I've review the template code, Template:WPBannerMeta, Template:WPBannerMeta/core and Template:WPBannerMeta/taskforce on zhwiki need to be updated to deploy. I've tested the changes in sandboxes (Template:WPBannerMeta/sandbox, Template:WPBannerMeta/core/sandbox and Template:WPBannerMeta/taskforce/sandbox) , and it worked (see Template:WPBannerMeta/testcases).
Sorry for my poorly worded English.

Change 879832 merged by jenkins-bot:

[mediawiki/extensions/PageAssessments@master] PageAssessments.i18n.alias.php: add alias for zh

https://gerrit.wikimedia.org/r/879832

I've review the template code …

Thanks! This looks great. Reviewing the code, I think you're only adding the {{#assessment}} parser function to WPBannerMeta/core and WPBannerMeta/taskforce – is that right? I just want to make sure.

I played around with the test cases and confirmed invalid class= values such as adsflkjdsafk do not store anything, and C, c and all work, so it does appear some normalization is being done.


Next question for you: I see that "task forces" (or "sub-WikiProjects") are a thing at zhwiki. You may wish to record assessments for task forces separately from the parent WikiProject. This means you can for instance have a bot report the most popular pages for a task force, in addition to the entire WikiProject. Let me know how you'd like to proceed there.

Mentioned in SAL (#wikimedia-operations) [2023-01-23T14:18:55Z] <taavi> mwscript extensions/WikimediaMaintenance/createExtensionTables.php --wiki=zhwiki pageassessments # T326387

Change 876196 merged by jenkins-bot:

[operations/mediawiki-config@master] zhwiki: Install PageAssessments

https://gerrit.wikimedia.org/r/876196

Mentioned in SAL (#wikimedia-operations) [2023-01-23T14:25:58Z] <taavi@deploy1002> Started scap: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]]

Mentioned in SAL (#wikimedia-operations) [2023-01-23T14:27:38Z] <taavi@deploy1002> stang and taavi: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-01-23T14:37:22Z] <taavi@deploy1002> Finished scap: Backport for [[gerrit:876196|zhwiki: Install PageAssessments (T326387)]] (duration: 11m 24s)

Extension installed, but further on-wiki configuration is needed.

Shizhao claimed this task.

On-wiki configuration done

We should use the same task for the whole deployment process.

We never heard back if zhwiki wanted $wgPageAssessmentsSubprojects set. I assume they do, since it seems to be a thing, i.e. https://zh.wikipedia.org/wiki/Template:WPBannerMeta/taskforce. Pinging @BlackShadowG again to confirm. See T326387#8533587

在T326387#8561353中,@MusikAnimal写道:

We should use the same task for the whole deployment process.

We never heard back if zhwiki wanted $wgPageAssessmentsSubprojects set. I assume they do, since it seems to be a thing, i.e. https://zh.wikipedia.org/wiki/Template:WPBannerMeta/taskforce. Pinging @BlackShadowG again to confirm. See T326387#8533587

Sorry for the late reply, task forces are a thing at zhwiki, it's currently recorded in {{#assessment}} like {{#assessment:<wikiproject name>/<task force name>}}, which is simliar to enwiki. It seems that $wgPageAssessmentsSubprojects need to be set on zhwiki.

Change 884474 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[operations/mediawiki-config@master] InitialiseSettings: add zhwiki to wgPageAssessmentsSubprojects

https://gerrit.wikimedia.org/r/884474

Sorry for the late reply, task forces are a thing at zhwiki, it's currently recorded in {{#assessment}} like {{#assessment:<wikiproject name>/<task force name>}}, which is simliar to enwiki. It seems that $wgPageAssessmentsSubprojects need to be set on zhwiki.

Thanks. It would have been most ideal to have made this change before deployment, but I know that isn't your fault :)

After r876196 is deployed, we'll need to invalidate the page cache again. I'm not sure if a null edit will do the trick or if you will need to undo/re-do your edits to the templates.

I have scheduled the deployment for 21:00 UTC on Monday, January 30.


It looks like zhwiki does have a Template:Class mask, which I assume like enwiki is used to normalize the values passed in for assessments. It is important to use this normalization before passing values to {{#assessment}}, otherwise you end up with corrupt and/or inaccurate data.

I played around with the test cases and confirmed invalid class= values such as adsflkjdsafk do not store anything, and C, c and all work, so it does appear some normalization is being done.

Indeed, it looks like there is still some corrupt data getting through:

mysql:research@dbstore1007.eqiad.wmnet [zhwiki]> SELECT pa_class, COUNT(pa_page_id) AS `count` FROM page_assessments GROUP BY pa_class ORDER BY `count` DESC;
+----------------------+--------+
| pa_class             | count  |
+----------------------+--------+
| 小作品               | 306355 |
|                      | 234063 |
| 初                   | 107655 |
| 分类                 |  60857 |
| 丙                   |  26577 |
| 模板                 |  20833 |
| 列表                 |  15862 |
| 乙                   |   8374 |
| 未评                 |   5951 |
| 优良                 |   5290 |
| 非条目               |   2949 |
| 消歧义               |   2119 |
| 典范                 |   2104 |
| 文件                 |   1413 |
| 主题                 |    935 |
| 特色列表             |    651 |
| 专题                 |    568 |
| 重定向               |    278 |
| 甲                   |    253 |
| 分類                 |    136 |
| 未来                 |     56 |
| 丙级列表             |     55 |
| 动态                 |     44 |
| 模块                 |     34 |
| 不适用               |     31 |
| 图像                 |     15 |
| 優良                 |     13 |
| -{zh-hans:文件;zh-   |      9 |
| 乙级列表             |      8 |
| 合并                 |      8 |
| 丙級列表             |      7 |
| 典範                 |      6 |
| 特色                 |      5 |
| 乙級列表             |      4 |
| 甲级列表             |      4 |
| 非條目               |      2 |
| current              |      2 |
| 消歧義               |      2 |
| 草稿                 |      1 |
+----------------------+--------+

If it helps you debug what's going on:

Similarly, here's the results for the importance level:

mysql:research@dbstore1007.eqiad.wmnet [zhwiki]> SELECT pa_importance, COUNT(pa_page_id) AS `count` FROM page_assessments GROUP BY pa_importance ORDER BY `count` DESC;
+---------------+--------+
| pa_importance | count  |
+---------------+--------+
| 未知          | 449334 |
| 低            | 152682 |
| 不适用        |  87466 |
| 中            |  49575 |
|               |  23560 |
| 高            |  21841 |
| 极低          |   7004 |
| 极高          |   6461 |
| 不適用        |   4634 |
| 无            |    663 |
| 極高          |    218 |
| 極低          |     66 |
| na            |     23 |
| 無            |      2 |
+---------------+--------+
14 rows in set (0.304 sec)

I recommend passing the classification and importance values through Template:Class mask and Template:Importance mask where you pass them to the {{#assessment}} parser function, i.e. Template:WPBannerMeta/core and Template:WPBannerMeta/taskforce should have:

{{#assessment:{{{PROJECT}}}|{{#ifeq:{{{class|¬}}}|¬||{{class mask|{{{class}}}}}}}|{{#ifeq:{{{importance|¬}}}|¬||{{importance mask|{{{importance}}}}}}}}}

You could make this change after we deploy https://gerrit.wikimedia.org/r/884474 tomorrow, that way it will reprocess all assessments.

在T326387#8567978中,@MusikAnimal写道:

Temporary fixed in Template:Class mask

I recommend passing the classification and importance values through Template:Class mask and Template:Importance mask where you pass them to the {{#assessment}} parser function, i.e. Template:WPBannerMeta/core and Template:WPBannerMeta/taskforce should have:

{{#assessment:{{{PROJECT}}}|{{#ifeq:{{{class|¬}}}|¬||{{class mask|{{{class}}}}}}}|{{#ifeq:{{{importance|¬}}}|¬||{{importance mask|{{{importance}}}}}}}}}

These templates have been updated

Change 884474 merged by jenkins-bot:

[operations/mediawiki-config@master] InitialiseSettings: add zhwiki to wgPageAssessmentsSubprojects

https://gerrit.wikimedia.org/r/884474

Mentioned in SAL (#wikimedia-operations) [2023-01-30T21:01:38Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:884474|InitialiseSettings: add zhwiki to wgPageAssessmentsSubprojects (T326387)]]

Mentioned in SAL (#wikimedia-operations) [2023-01-30T21:03:21Z] <urbanecm@deploy1002> urbanecm and musikanimal: Backport for [[gerrit:884474|InitialiseSettings: add zhwiki to wgPageAssessmentsSubprojects (T326387)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-01-30T21:21:29Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:884474|InitialiseSettings: add zhwiki to wgPageAssessmentsSubprojects (T326387)]] (duration: 19m 51s)

$wgPageAssessmentsSubprojects is now set for zhwiki, but unfortunately it will require a complete re-run to get the database to store the correct values (ref). All the current task forces are already stored, so PageAssessments doesn't know that anything about them has changed – in this case the ID of the parent WikiProject.

I suppose the best way to do fix this is undo the edits to the templates, wait a day or so, then add them back. It's important to wait and not re-add the {{#assessment}} calls immediately, since we need to first remove all WikiProjects from the database. I have global-interface-editing rights so I can take care of this, if you'd like @Shizhao @BlackShadowG ? Alternatively, please ping me before re-adding the {{#assessment}} parser function so I can make sure the database is clean.

While testing, I also discovered some WikiProjects aren't using Template:WPBannerMeta, and thus the assessments are not getting stored. An example is https://zh.wikipedia.org/wiki/Template:WikiProject_Video_games. You could alternatively put the {{#assessment}} parser function in that template, but I think it makes more sense to keep it centralized and to have all WikiProjects templates use Template:WPBannerMeta. Pinging to bring this to your attention, should you want to do anything about it: .

在T326387#8572718中,@MusikAnimal写道:

I suppose the best way to do fix this is undo the edits to the templates, wait a day or so, then add them back. It's important to wait and not re-add the {{#assessment}} calls immediately, since we need to first remove all WikiProjects from the database. I have global-interface-editing rights so I can take care of this, if you'd like @Shizhao @BlackShadowG ? Alternatively, please ping me before re-adding the {{#assessment}} parser function so I can make sure the database is clean.

If you can modify the relevant templates, then it is better for you to modify

While testing, I also discovered some WikiProjects aren't using Template:WPBannerMeta, and thus the assessments are not getting stored. An example is https://zh.wikipedia.org/wiki/Template:WikiProject_Video_games. You could alternatively put the {{#assessment}} parser function in that template, but I think it makes more sense to keep it centralized and to have all WikiProjects templates use Template:WPBannerMeta. Pinging to bring this to your attention, should you want to do anything about it: .

The WP template that does not use {{#assessment}}, if it is used in the future, will there be any technical problems?

If you can modify the relevant templates, then it is better for you to modify

Just an update that I haven't forgotten about this! We've just been really busy with the Community Wishlist Survey. I'm hoping to take care of this today but certainly this week :)

The WP template that does not use {{#assessment}}, if it is used in the future, will there be any technical problems?

No, adding {{#assessment}} to the template will cause it to be stored for all pages that transclude it, so that should work fine. If there are task forces under that WikiProject, they should be stored properly because https://gerrit.wikimedia.org/r/884474 has been deployed. So you can go ahead and add it now if you want to (or change the template to use Template:WPBannerMeta). This would actually be interesting to see, because if I see task forces in the database then we know that part works. I'm pretty confident it will work just fine, but what I said at T326387#8572718 is actually only a theory after inspecting the source code.

As you saw I have removed the {{#assessment}} calls, but the propagation is taking a lot longer than expected. I suspect it may be at least one more day before we can add back the parser functions. Note also I will need to run a maintenance script first, so I'll be handling re-adding the parser functions, too. I hope to have all this done by the end of the week.

MusikAnimal set the point value for this task to 5.Feb 8 2023, 5:45 AM

Mentioned in SAL (#wikimedia-operations) [2023-02-09T02:23:37Z] <TheresNoTime> [samtar@mwmaint1002 ~]$ mwscript extensions/PageAssessments/maintenance/purgeUnusedProjects.php --wiki zhwiki --dry-run for T326387

Mentioned in SAL (#wikimedia-operations) [2023-02-09T02:28:29Z] <TheresNoTime> [samtar@mwmaint1002 ~]$ mwscript extensions/PageAssessments/maintenance/purgeUnusedProjects.php --wiki zhwiki for T326387

[samtar@mwmaint1002 ~]$ mwscript extensions/PageAssessments/maintenance/purgeUnusedProjects.php --wiki zhwiki

*******************************************************************************
NOTE: Do not run maintenance scripts directly, use maintenance/run.php instead!
      Running scripts directly has been deprecated in MediaWiki 1.40.
      It may not work for some (or any) scripts in the future.
*******************************************************************************

Projects before purge: 479
Purging unused projects from page_assessments_projects...
Done.
Projects after purge: 7
MusikAnimal closed this task as Resolved.EditedFeb 9 2023, 2:53 AM

Alright, we're all set :) I've added back the parser function to the templates, and slowly but surely all assessments will get restored. I have already observed that task forces are correctly being stored this time.

Projects before purge: 479
Purging unused projects from page_assessments_projects...
Done.
Projects after purge: 7

In case anyone was wondering, those remaining 7 were not task forces, so there's no issue. This did however identify a bug in PageAssessments, where when a page is moved without redirect, the assessments for the old location don't get removed. I'll file a task to get that fixed.

Next, I will get zhwiki added to XTools, which will make the assessments show up in the ArticleInfo gadget as well as https://pageviews.wmcloud.org and probably other places, too. I will do this when I return from holiday later this month.

Thanks for your patience! Resolving.

Change 901655 had a related patch set uploaded (by Winston Sung; author: Winston Sung):

[mediawiki/extensions/PageAssessments@master] PageAssessments: Magic word aliases should not store in zh

https://gerrit.wikimedia.org/r/901655

Change 901655 merged by jenkins-bot:

[mediawiki/extensions/PageAssessments@master] PageAssessments: Special page aliases should not store in zh

https://gerrit.wikimedia.org/r/901655