Page MenuHomePhabricator
Create Task
Maniphest T203184

Deploy PageTriage AfC to production
Closed, ResolvedPublic
Actions

Description

The goal of this task is to list and track the steps needed to deploy PageTriage AfC to English Wikipedia (enwiki).

The features to be deployed are:

  1. NPP/AfC radio buttons
  2. AfC state filters (NOT ORES or Copyvio)
  3. AfC specific sort options.

These features are controlled by the $wgPageTriageDraftNamespaceId configuration variable.

The steps needed are

  1. Deploy config patch to set $wgPageTriageDraftNamespaceId=118
  2. Run PageTriage/maintenance/populateDraftQueue.php for enwiki

Doing the steps in this order result in a short time where the AfC option is visible but the queue is getting progressively filled up.

Doing the steps in the opposite order would be worst since it would add drafts to the queue but any change to those drafts would not refresh their metadata since the wgPageTriageDraftNamespaceId being false would prevent it. It would lead to some metadata being out of date, which is hard to detect. The populateDraftQueue.php script is not built to be re-run to fix those.

Tentative release date is Monday September 17 2018

Details

SubjectRepoBranchLines +/-
Maintenance script to delete afc_statesmediawiki/extensions/PageTriagewmf/1.32.0-wmf.22+74 -0
Find pages in queue that don't have afc_statemediawiki/extensions/PageTriagewmf/1.32.0-wmf.22+12 -3
Maintenance script to delete afc_statesmediawiki/extensions/PageTriagemaster+74 -0
Find pages in queue that don't have afc_statemediawiki/extensions/PageTriagemaster+12 -3
Find pages in queue that don't have afc_statemediawiki/extensions/PageTriagewmf/1.32.0-wmf.20+12 -3
Enable PageTriage AfC on enwikioperations/mediawiki-configmaster+1 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

SBisson created this task.Aug 30 2018, 6:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 30 2018, 6:30 PM
kostajh set Due Date to Sep 17 2018, 4:00 AM.Aug 30 2018, 6:45 PM
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptAug 30 2018, 6:45 PM
MMiller_WMF added a parent task: T196177: New Pages Feed: draft addition.Aug 30 2018, 7:54 PM
MMiller_WMF mentioned this in T201087: New Pages Feed: run draft backfill script in English Wikipedia.
MMiller_WMF moved this task from Inbox to Upcoming Work on the Growth-Team board.
SBisson merged a task: T201087: New Pages Feed: run draft backfill script in English Wikipedia.Sep 5 2018, 12:50 PM
SBisson added subscribers: SQL, Ammarpad, Keegan and 7 others.
gerritbot subscribed.Sep 6 2018, 8:17 PM

Change 458583 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[operations/mediawiki-config@master] Enable PageTriage AfC on enwiki

https://gerrit.wikimedia.org/r/458583

gerritbot added a project: Patch-For-Review.Sep 6 2018, 8:17 PM
SBisson claimed this task.Sep 6 2018, 8:18 PM
JTannerWMF moved this task from Upcoming Work to Sprint 0 (Growth Team) on the Growth-Team board.Sep 17 2018, 5:41 PM
JTannerWMF edited projects, added Growth-Team (Sprint 0 (Growth Team)); removed Growth-Team.
JTannerWMF moved this task from Incoming to In Progress on the Growth-Team (Sprint 0 (Growth Team)) board.Sep 17 2018, 5:44 PM
gerritbot added a comment.Sep 17 2018, 6:37 PM

Change 458583 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable PageTriage AfC on enwiki

https://gerrit.wikimedia.org/r/458583

Stashbot subscribed.Sep 17 2018, 6:40 PM

Mentioned in SAL (#wikimedia-operations) [2018-09-17T18:40:56Z] <sbisson@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T203184 (duration: 00m 50s)

gerritbot added a comment.Sep 17 2018, 7:25 PM

Change 460952 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@master] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/460952

SBisson added a comment.Sep 18 2018, 10:40 AM

It turns out we forgot 2 important steps when we first tried this deployment:

  1. create the afc_state tag in pagetriage_tags
  2. invalidate the tags cache

We eventually did those steps and restarted by the populate script but it had already processed about 3600 pages. Those have metadata but no afc state. The populate script was updated (patch above) to take that into account. It will have to be re-run to fix those pages.

MMiller_WMF added a subtask: T204629: AfC: Issues with reloading the page .Sep 18 2018, 5:24 PM
gerritbot added a comment.Sep 18 2018, 5:49 PM

Change 460952 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/460952

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-09-25 (1.32.0-wmf.23)).Sep 18 2018, 6:00 PM
gerritbot added a comment.Sep 18 2018, 6:25 PM

Change 461178 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.20] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/461178

gerritbot added a comment.Sep 19 2018, 6:42 PM

Change 461178 abandoned by Sbisson:
Find pages in queue that don't have afc_state

Reason:
Will cherry pick to wmf.22 instead

https://gerrit.wikimedia.org/r/461178

gerritbot added a comment.Sep 19 2018, 6:42 PM

Change 461450 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.22] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/461450

gerritbot added a comment.Sep 19 2018, 7:14 PM

Change 461455 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@master] Maintenance script to delete afc_states

https://gerrit.wikimedia.org/r/461455

gerritbot added a comment.Sep 19 2018, 7:39 PM

Change 461455 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@master] Maintenance script to delete afc_states

https://gerrit.wikimedia.org/r/461455

gerritbot added a comment.Sep 19 2018, 8:04 PM

Change 461465 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.22] Maintenance script to delete afc_states

https://gerrit.wikimedia.org/r/461465

• TBolliger unsubscribed.Sep 19 2018, 8:13 PM
gerritbot added a comment.Sep 19 2018, 11:09 PM

Change 461450 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.22] Find pages in queue that don't have afc_state

https://gerrit.wikimedia.org/r/461450

gerritbot added a comment.Sep 19 2018, 11:10 PM

Change 461465 merged by jenkins-bot:
[mediawiki/extensions/PageTriage@wmf/1.32.0-wmf.22] Maintenance script to delete afc_states

https://gerrit.wikimedia.org/r/461465

Stashbot added a comment.Sep 19 2018, 11:17 PM

Mentioned in SAL (#wikimedia-operations) [2018-09-19T23:17:48Z] <catrope@deploy1001> Synchronized php-1.32.0-wmf.22/extensions/PageTriage/: PageTriage fixes for T203184 (duration: 00m 58s)

ReleaseTaggerBot edited projects, added MW-1.32-notes (WMF-deploy-2018-09-18 (1.32.0-wmf.22)); removed MW-1.32-notes (WMF-deploy-2018-09-25 (1.32.0-wmf.23)).Sep 20 2018, 12:00 AM
Stashbot added a comment.Sep 20 2018, 3:11 PM

Mentioned in SAL (#wikimedia-operations) [2018-09-20T15:11:37Z] <stephanebisson> Starting mwscript extensions/PageTriage/maintenance/DeleteAfcStates.php --wiki enwiki (T203184)

Stashbot added a comment.Sep 20 2018, 3:14 PM

Mentioned in SAL (#wikimedia-operations) [2018-09-20T15:14:17Z] <stephanebisson> Finished mwscript extensions/PageTriage/maintenance/DeleteAfcStates.php --wiki enwiki (T203184)

Stashbot added a comment.Sep 20 2018, 3:15 PM

Mentioned in SAL (#wikimedia-operations) [2018-09-20T15:15:05Z] <stephanebisson> Starting mwscript extensions/PageTriage/maintenance/populateDraftQueue.php --wiki enwiki (T203184)

Etonkovidova closed subtask T204629: AfC: Issues with reloading the page as Resolved.Sep 20 2018, 8:56 PM
SBisson removed Due Date.Sep 21 2018, 5:32 PM
Restricted Application changed the subtype of this task from "Deadline" to "Task". · View Herald TranscriptSep 21 2018, 5:32 PM
SBisson moved this task from In Progress to QA on the Growth-Team (Sprint 0 (Growth Team)) board.Sep 21 2018, 5:32 PM
Etonkovidova added a comment.Sep 21 2018, 9:19 PM

@SBisson There is always a discrepancy between the number of drafts with afc_state in db and the number with AfC 'All' filter. Last time I checked the difference was about 8,000. Any ideas why such discrepancy is happening?

MaxSem unsubscribed.Sep 21 2018, 9:28 PM
SBisson added a comment.Sep 24 2018, 10:48 AM
In T203184#4607301, @Etonkovidova wrote:

@SBisson There is always a discrepancy between the number of drafts with afc_state in db and the number with AfC 'All' filter. Last time I checked the difference was about 8,000. Any ideas why such discrepancy is happening?

There could be a small discrepancy because the data is changing and the compilation process can be a bit delayed but 8000 would be too much to be explained by those.

This is what I see right now:
Number of drafts reported by Special:NewPagesFeed: 41876

Number of drafts with afc_state in the db:

mysql:research@s3-analytics-slave [enwiki]> select count(ptrp_page_id) from pagetriage_page inner join page on ptrp_page_id=page_id inner join pagetriage_page_tags on ptrp_page_id=ptrpt_page_id and ptrpt_tag_id=20 where page_namespace=118 and page_is_redirect=0;
+---------------------+
| count(ptrp_page_id) |
+---------------------+
|               41972 |
+---------------------+
1 row in set (1.61 sec)
MMiller_WMF added a subtask: T205324: [wmf.22] AfC - Sort option 'Created date(newest)' displayed for default/reload.Sep 24 2018, 8:13 PM
Etonkovidova closed this task as Resolved.Sep 24 2018, 11:34 PM

Thx, @SBisson - the numbers look much closer now (I did not exclude page_is_redirect=0).

Another question - there are few drafts that should have ptrpt_tag_id and apparently they don't have it. I used this query:

[enwiki]> select count(*) from page where page_namespace=118 and page_is_redirect=0 and  page_id not in (select  ptrpt_page_id  from pagetriage_page_tags where ptrpt_tag_id=20); 
+----------+
| count(*) |
+----------+
|       11 |
+----------+

I checked some of those pages - they all have AFCH templates.

SBisson removed subtasks: T205324: [wmf.22] AfC - Sort option 'Created date(newest)' displayed for default/reload, T204889: New Pages Feed: fix AfC submitted/declined dates, T204629: AfC: Issues with reloading the page .Sep 25 2018, 10:23 AM
SBisson added a comment.Sep 25 2018, 12:56 PM
In T203184#4613107, @Etonkovidova wrote:

Another question - there are few drafts that should have ptrpt_tag_id and apparently they don't have it. I used this query:

All those pages not only don't have afc_state tags, they are simply NOT in the PageTriage queue.

I re-ran the script and 19 were added. The last one, Draft:Dennis_goldstein, is deleted so it won't ever be in PageTriage.

I don't know if they were missed the first time or if something happen for them to drop out of the queue but we should keep an eye on that and re-check in a few days.

DannyS712 moved this task from Backlog to PageCuration & NewPagesFeed on the English-Wikipedia-New-Pages-Patrol board.Dec 2 2019, 9:12 PM
Maintenance_bot removed a project: Patch-For-Review.Dec 2 2019, 10:10 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL