Page MenuHomePhabricator

Deploy PageTriage AfC to production
Closed, ResolvedPublic

Description

The goal of this task is to list and track the steps needed to deploy PageTriage AfC to English Wikipedia (enwiki).

The features to be deployed are:

  1. NPP/AfC radio buttons
  2. AfC state filters (NOT ORES or Copyvio)
  3. AfC specific sort options.

These features are controlled by the $wgPageTriageDraftNamespaceId configuration variable.

The steps needed are

  1. Deploy config patch to set $wgPageTriageDraftNamespaceId=118
  2. Run PageTriage/maintenance/populateDraftQueue.php for enwiki

Doing the steps in this order result in a short time where the AfC option is visible but the queue is getting progressively filled up.

Doing the steps in the opposite order would be worst since it would add drafts to the queue but any change to those drafts would not refresh their metadata since the wgPageTriageDraftNamespaceId being false would prevent it. It would lead to some metadata being out of date, which is hard to detect. The populateDraftQueue.php script is not built to be re-run to fix those.

Tentative release date is Monday September 17 2018

Event Timeline

SBisson created this task.Aug 30 2018, 6:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 30 2018, 6:30 PM
kostajh set Due Date to Sep 17 2018, 4:00 AM.Aug 30 2018, 6:45 PM
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptAug 30 2018, 6:45 PM

Change 458583 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[operations/mediawiki-config@master] Enable PageTriage AfC on enwiki

https://gerrit.wikimedia.org/r/458583

SBisson claimed this task.Sep 6 2018, 8:18 PM

Change 458583 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable PageTriage AfC on enwiki

https://gerrit.wikimedia.org/r/458583

Mentioned in SAL (#wikimedia-operations) [2018-09-17T18:40:56Z] <sbisson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T203184 (duration: 00m 50s)

Change 460952 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@master] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/460952

It turns out we forgot 2 important steps when we first tried this deployment:

  1. create the afc_state tag in pagetriage_tags
  2. invalidate the tags cache

We eventually did those steps and restarted by the populate script but it had already processed about 3600 pages. Those have metadata but no afc state. The populate script was updated (patch above) to take that into account. It will have to be re-run to fix those pages.

Change 460952 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/460952

Change 461178 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.20] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/461178

Change 461178 abandoned by Sbisson:
Find pages in queue that don't have afc_state

Reason:
Will cherry pick to wmf.22 instead

https://gerrit.wikimedia.org/r/461178

Change 461450 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.22] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/461450

Change 461455 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@master] Maintenance script to delete afc_states

https://gerrit.wikimedia.org/r/461455

Change 461455 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Maintenance script to delete afc_states

https://gerrit.wikimedia.org/r/461455

Change 461465 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.22] Maintenance script to delete afc_states

https://gerrit.wikimedia.org/r/461465

Change 461450 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.22] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/461450

Change 461465 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.22] Maintenance script to delete afc_states

https://gerrit.wikimedia.org/r/461465

Mentioned in SAL (#wikimedia-operations) [2018-09-19T23:17:48Z] <catrope@deploy1001> Synchronized php-1.32.0-wmf.22/extensions/PageTriage/: PageTriage fixes for T203184 (duration: 00m 58s)

Mentioned in SAL (#wikimedia-operations) [2018-09-20T15:11:37Z] <stephanebisson> Starting mwscript extensions/PageTriage/maintenance/DeleteAfcStates.php --wiki enwiki (T203184)

Mentioned in SAL (#wikimedia-operations) [2018-09-20T15:14:17Z] <stephanebisson> Finished mwscript extensions/PageTriage/maintenance/DeleteAfcStates.php --wiki enwiki (T203184)

Mentioned in SAL (#wikimedia-operations) [2018-09-20T15:15:05Z] <stephanebisson> Starting mwscript extensions/PageTriage/maintenance/populateDraftQueue.php --wiki enwiki (T203184)

SBisson removed Due Date.Sep 21 2018, 5:32 PM
Restricted Application changed the subtype of this task from "Deadline" to "Task". · View Herald TranscriptSep 21 2018, 5:32 PM

@SBisson There is always a discrepancy between the number of drafts with afc_state in db and the number with AfC 'All' filter. Last time I checked the difference was about 8,000. Any ideas why such discrepancy is happening?

MaxSem removed a subscriber: MaxSem.Sep 21 2018, 9:28 PM

@SBisson There is always a discrepancy between the number of drafts with afc_state in db and the number with AfC 'All' filter. Last time I checked the difference was about 8,000. Any ideas why such discrepancy is happening?

There could be a small discrepancy because the data is changing and the compilation process can be a bit delayed but 8000 would be too much to be explained by those.

This is what I see right now:
Number of drafts reported by Special:NewPagesFeed: 41876

Number of drafts with afc_state in the db:

mysql:research@s3-analytics-slave [enwiki]> select count(ptrp_page_id) from pagetriage_page inner join page on ptrp_page_id=page_id inner join pagetriage_page_tags on ptrp_page_id=ptrpt_page_id and ptrpt_tag_id=20 where page_namespace=118 and page_is_redirect=0;
+---------------------+
| count(ptrp_page_id) |
+---------------------+
|               41972 |
+---------------------+
1 row in set (1.61 sec)
Etonkovidova closed this task as Resolved.Sep 24 2018, 11:34 PM

Thx, @SBisson - the numbers look much closer now (I did not exclude page_is_redirect=0).

Another question - there are few drafts that should have ptrpt_tag_id and apparently they don't have it. I used this query:

[enwiki]> select count(*) from page where page_namespace=118 and page_is_redirect=0 and  page_id not in (select  ptrpt_page_id  from pagetriage_page_tags where ptrpt_tag_id=20); 
+----------+
| count(*) |
+----------+
|       11 |
+----------+

I checked some of those pages - they all have AFCH templates.

Another question - there are few drafts that should have ptrpt_tag_id and apparently they don't have it. I used this query:

All those pages not only don't have afc_state tags, they are simply NOT in the PageTriage queue.

I re-ran the script and 19 were added. The last one, Draft:Dennis_goldstein, is deleted so it won't ever be in PageTriage.

I don't know if they were missed the first time or if something happen for them to drop out of the queue but we should keep an eye on that and re-check in a few days.