Page MenuHomePhabricator

Tracking: Direct live production traffic at Parsoid/PHP
Closed, ResolvedPublic

Description

This is just a tracking task with blocking subtasks to track all the work that needs to happen before we can deploy to production

Here is the current deployment timeline:

  • Oct 29 - Nov 4: Parsoid/PHP deployed to the Parsoid cluster to process all changeprop traffic (mirrored from Parsoid/JS) from 10% to 100%.
  • Nov 19: Parsoid/PHP deployed to testwiki and test2wki (VE, Content Translation use Parsoid/PHP).
  • Nov 21: Parsoid/PHP deployed to mediawiki.org (VE uses Parsoid/PHP).
  • Nov25: Parsoid/PHP to be deployed to all private wikis (VE, Flow, Content Translation use Parsoid/PHP). An attempt on Nov 21 was rolled back because of T238849.
  • Nov 26: Enable Parsoid/PHP use for Flow on mediawiki.org
  • Nov 26: Parsoid/PHP to be deployed to group 0 and group 1 wikis (VE, Content Translation , Media Content Service use Parsoid/PHP). We have effectively deployed to all group 0 wikis since all the other closed wikis will have no impact from the switchover.
  • Dec 2: Flow on all group 0 and 1 wikis use Parsoid/PHP

Deployments have been really smooth so far, so, we'll probably finish group 2 wiki deployment on Dec 2 in multiple steps within a few hours:

  • Step 1: Enable Parsoid/PHP on zh, ru, sr wikis. This lets us exercise language variant code. We need MCS / Android product folks ready to test and verify. Based on crashers in production for language variant requests, we reverted the switchover on zh, sr wikis.
  • Step 2: Direct all language variant requests to Parsoid/JS always. This will let us fix the crashers and performance on it separately. Re-enable Parsoid/PHP on sr and zh wikis.
  • Step 3: Enable Parsoid/PHP on everything else but enwiki. Maybe wait for about an hour or less.
  • Step 4: Enable Parsoid/PHP on enwiki.
  • Step 5: Enable Flow everywhere (group 0 and group 1 previously)

At this point, Parsoid/PHP is enabled for all products except Linter.

  • Enabling linter posts from Parsoid/PHP (which will temporarily introduce duplicate lint posts -- verify we do the right thing) and disable linting from Parsoid/JS
  • Direct language variant requests to Parsoid/PHP -- completed Dec 18

At this point, Parsoid/JS will be operational in mirror / shadow mode for the really unlikely possibility of need to switch back to Parsoid/JS for some product somewhere.

  • Create a phab task to decommission Parsoid/JS (T241207)

Details

Related Gerrit Patches:
operations/mediawiki-config : masterParsoid VRS: Switch all clients to Parsoid/PHP
mediawiki/services/restbase/deploy : masterSwitch everything to Parsoid/PHP
mediawiki/services/restbase/deploy : masterSwitch everything but enwiki to Parsoid/PHP
mediawiki/services/restbase/deploy : masterSwitch sr and zh wp back to Parsoid/PHP
mediawiki/services/restbase/deploy : masterSwitch ru, sr and zh wikipedias to Parsoid/PHP
operations/mediawiki-config : masterParsoid: Switch groups 0 and 1 to Parsoid/PHP
mediawiki/services/restbase/deploy : masterSwitch group 0 and 1 to Parsoid/PHP
operations/mediawiki-config : masterParsoid: Switch mw.org to Parsoid/PHP
operations/mediawiki-config : masterParsoid: Switch private wiki consumers (Flow, VE) to Parsoid/PHP
operations/mediawiki-config : masterParsoid VRS: Switch back private wikis to Parsoid/JS
operations/mediawiki-config : masterParsoid/PHP Service: Use HTTPS
operations/mediawiki-config : masterVirtualRESTService: Switch private wikis to Parsoid/PHP
mediawiki/services/restbase/deploy : masterSwitch mediawiki.org to Parsoid/PHP
mediawiki/core : wmf/1.35.0-wmf.5Parsoid VRS: Add the Host header
mediawiki/core : masterParsoid VRS: Add the Host header
mediawiki/services/restbase/deploy : masterSwitch all of Beta to Parsoid/PHP
mediawiki/services/restbase/deploy : masterSwitch test.wp and test2.wp to Parsoid/PHP only
mediawiki/services/parsoid/deploy : masterRestart fpm during the promote stage
operations/mediawiki-config : masterSet up a multiversion rest.php endpoint

Related Objects

StatusSubtypeAssignedTask
Resolvedssastry
Resolvedmobrovac
DeclinedNone
ResolvedEevans
OpenEvanProdromou
DeclinedEvanProdromou
ResolvedTgr
ResolvedEvanProdromou
ResolvedPchelolo
OpenEvanProdromou
Resolvedtstarling
Resolvedtstarling
ResolvedEvanProdromou
Resolvedtstarling
ResolvedAnomie
ResolvedTgr
DuplicateNone
ResolvedEvanProdromou
ResolvedEvanProdromou
Openapaskulin
Resolvedtstarling
ResolvedEvanProdromou
ResolvedTgr
ResolvedTgr
ResolvedTgr
ResolvedBPirkle
ResolvedBPirkle
DeclinedEvanProdromou
DeclinedEvanProdromou
ResolvedEvanProdromou
ResolvedTgr
Resolvedsbassett
Resolvedkostajh
Resolvedssastry
Resolvedssastry
Resolvedssastry
DuplicateArlolra
Resolvedssastry
ResolvedGoalssastry
Resolvedssastry
ResolvedJoe
Resolvedssastry
Resolvedssastry
ResolvedArlolra
ResolvedDzahn
Resolvedssastry
ResolvedNone
ResolvedBUG REPORTssastry
Resolvedssastry
Resolvedssastry
Resolvedssastry
DuplicateNone
Resolvedcscott
Resolvedssastry
Resolvedmobrovac
ResolvedDzahn
ResolvedJoe
ResolvedDzahn
Resolvedmobrovac
DuplicateNone
Resolvedmobrovac
Resolvedmobrovac
Resolvedssastry
ResolvedPchelolo
ResolvedPchelolo
ResolvedPchelolo
Opencscott
Resolvedcscott
Resolvedcscott
Resolvedssastry
Resolvedcscott
Opencscott
OpenNone
ResolvedSbailey
Resolvedmobrovac
ResolvedArlolra
InvalidSpikeNone
Resolvedmobrovac
Resolvedssastry
ResolvedArlolra
Resolvedmobrovac
Resolvedssastry
Resolvedssastry
Resolvedcscott
DuplicateArlolra

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
ssastry updated the task description. (Show Details)Nov 24 2019, 6:24 PM

Change 552845 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/mediawiki-config@master] Parsoid: Switch private wiki consumers (Flow, VE) to Parsoid/PHP

https://gerrit.wikimedia.org/r/552845

Change 552845 merged by jenkins-bot:
[operations/mediawiki-config@master] Parsoid: Switch private wiki consumers (Flow, VE) to Parsoid/PHP

https://gerrit.wikimedia.org/r/552845

Mentioned in SAL (#wikimedia-operations) [2019-11-25T17:50:12Z] <mobrovac@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Parsoid: Switch private wiki clients (Flow, VE) to Parsoid/PHP -- T229015 (duration: 00m 53s)

ssastry updated the task description. (Show Details)Nov 25 2019, 11:22 PM

Change 552965 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/mediawiki-config@master] Parsoid: Switch mw.org to Parsoid/PHP

https://gerrit.wikimedia.org/r/552965

Change 552965 merged by jenkins-bot:
[operations/mediawiki-config@master] Parsoid: Switch mw.org to Parsoid/PHP

https://gerrit.wikimedia.org/r/552965

Mentioned in SAL (#wikimedia-operations) [2019-11-26T07:53:04Z] <mobrovac@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Parsoid: Switch Flow to Parsoid/PHP on mw.org -- T229015 (duration: 00m 52s)

mobrovac updated the task description. (Show Details)Nov 26 2019, 7:57 AM

Change 552972 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/mediawiki-config@master] Parsoid: Switch groups 0 and 1 to Parsoid/PHP

https://gerrit.wikimedia.org/r/552972

Change 553034 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/restbase/deploy@master] Switch group 0 and 1 to Parsoid/PHP

https://gerrit.wikimedia.org/r/553034

Change 553034 merged by Ppchelko:
[mediawiki/services/restbase/deploy@master] Switch group 0 and 1 to Parsoid/PHP

https://gerrit.wikimedia.org/r/553034

Mentioned in SAL (#wikimedia-operations) [2019-11-26T17:10:06Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@0b74625]: Switch group 0 and 1 to Parsoid-PHP T229015

Mentioned in SAL (#wikimedia-operations) [2019-11-26T17:25:43Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@0b74625]: Switch group 0 and 1 to Parsoid-PHP T229015 (duration: 15m 38s)

ssastry updated the task description. (Show Details)Nov 26 2019, 9:27 PM
ssastry updated the task description. (Show Details)Nov 27 2019, 5:29 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-02T12:59:31Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@eedba38]: Parsoid Proxy: Fixes - T229015

Change 554056 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/restbase/deploy@master] Switch ru, sr and zh wikipedias to Parsoid/PHP

https://gerrit.wikimedia.org/r/554056

Mentioned in SAL (#wikimedia-operations) [2019-12-02T13:14:20Z] <mobrovac@deploy1001> Finished deploy [restbase/deploy@eedba38]: Parsoid Proxy: Fixes - T229015 (duration: 14m 49s)

Change 552972 merged by jenkins-bot:
[operations/mediawiki-config@master] Parsoid: Switch groups 0 and 1 to Parsoid/PHP

https://gerrit.wikimedia.org/r/552972

Mentioned in SAL (#wikimedia-operations) [2019-12-02T15:38:27Z] <mobrovac@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Parsoid VRS: Switch groups 0 and 1 to Parsoid/PHP - T229015 (duration: 00m 59s)

ssastry updated the task description. (Show Details)Dec 2 2019, 3:54 PM

Change 554056 merged by Mobrovac:
[mediawiki/services/restbase/deploy@master] Switch ru, sr and zh wikipedias to Parsoid/PHP

https://gerrit.wikimedia.org/r/554056

Mentioned in SAL (#wikimedia-operations) [2019-12-02T16:27:45Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@3516382]: Switch ru, sr and zh wikipediae to Parsoid/PHP - T229015

Mentioned in SAL (#wikimedia-operations) [2019-12-02T16:41:38Z] <mobrovac@deploy1001> Finished deploy [restbase/deploy@3516382]: Switch ru, sr and zh wikipediae to Parsoid/PHP - T229015 (duration: 13m 53s)

Change 554112 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/restbase/deploy@master] Switch sr and zh wp back to Parsoid/PHP

https://gerrit.wikimedia.org/r/554112

Change 554112 merged by Mobrovac:
[mediawiki/services/restbase/deploy@master] Switch sr and zh wp back to Parsoid/PHP

https://gerrit.wikimedia.org/r/554112

Mentioned in SAL (#wikimedia-operations) [2019-12-02T17:42:43Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@ff7862f]: Switch sr and zh wikipediae back to Parsoid/JS - T229015

Mentioned in SAL (#wikimedia-operations) [2019-12-02T17:56:49Z] <mobrovac@deploy1001> Finished deploy [restbase/deploy@ff7862f]: Switch sr and zh wikipediae back to Parsoid/JS - T229015 (duration: 14m 06s)

Paladox added a subscriber: Paladox.Dec 2 2019, 5:58 PM
ssastry updated the task description. (Show Details)Dec 2 2019, 6:38 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-02T18:50:04Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@6a24685]: Parsoid Proxy: Direct html2html traffic to JS; Stop honouring the variant header; Switch sr and zh wikis to PHP - T229015

Mentioned in SAL (#wikimedia-operations) [2019-12-02T19:04:15Z] <mobrovac@deploy1001> Finished deploy [restbase/deploy@6a24685]: Parsoid Proxy: Direct html2html traffic to JS; Stop honouring the variant header; Switch sr and zh wikis to PHP - T229015 (duration: 14m 11s)

ssastry updated the task description. (Show Details)Dec 2 2019, 7:07 PM

Change 554129 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/restbase/deploy@master] Switch everything but enwiki to Parsoid/PHP

https://gerrit.wikimedia.org/r/554129

Change 554129 merged by Mobrovac:
[mediawiki/services/restbase/deploy@master] Switch everything but enwiki to Parsoid/PHP

https://gerrit.wikimedia.org/r/554129

Mentioned in SAL (#wikimedia-operations) [2019-12-02T19:23:39Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@e69e2e5]: Switch everything but enwiki to Parsoid/PHP - T229015

ssastry updated the task description. (Show Details)Dec 2 2019, 7:29 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-02T19:37:27Z] <mobrovac@deploy1001> Finished deploy [restbase/deploy@e69e2e5]: Switch everything but enwiki to Parsoid/PHP - T229015 (duration: 13m 48s)

Change 554136 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/restbase/deploy@master] Switch everything to Parsoid/PHP

https://gerrit.wikimedia.org/r/554136

ssastry updated the task description. (Show Details)Dec 2 2019, 7:58 PM

Change 554136 merged by Mobrovac:
[mediawiki/services/restbase/deploy@master] Switch everything to Parsoid/PHP

https://gerrit.wikimedia.org/r/554136

Mentioned in SAL (#wikimedia-operations) [2019-12-02T20:06:58Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@92acf1e]: Switch everything to Parsoid/PHP - T229015

Mentioned in SAL (#wikimedia-operations) [2019-12-02T20:21:57Z] <mobrovac@deploy1001> Finished deploy [restbase/deploy@92acf1e]: Switch everything to Parsoid/PHP - T229015 (duration: 14m 59s)

Change 554146 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/mediawiki-config@master] Parsoid VRS: Switch all clients to Parsoid/PHP

https://gerrit.wikimedia.org/r/554146

Change 554146 merged by Mobrovac:
[operations/mediawiki-config@master] Parsoid VRS: Switch all clients to Parsoid/PHP

https://gerrit.wikimedia.org/r/554146

Mentioned in SAL (#wikimedia-operations) [2019-12-02T20:36:14Z] <mobrovac@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Switch Flow on all wikis to Parsoid/PHP - T229015 (duration: 00m 59s)

ssastry updated the task description. (Show Details)Dec 2 2019, 8:38 PM

Mentioned in SAL (#wikimedia-operations) [2019-12-03T11:58:02Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@b346ebf]: Mirror html2html traffic to Parsoid/PHP - T229015 T239643

Mentioned in SAL (#wikimedia-operations) [2019-12-03T11:58:10Z] <mobrovac@deploy1001> deploy aborted: Mirror html2html traffic to Parsoid/PHP - T229015 T239643 (duration: 00m 00s)

Mentioned in SAL (#wikimedia-operations) [2019-12-03T11:58:33Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@b346ebf]: Mirror html2html traffic to Parsoid/PHP - T229015 T239643

Mentioned in SAL (#wikimedia-operations) [2019-12-03T12:12:02Z] <mobrovac@deploy1001> Finished deploy [restbase/deploy@b346ebf]: Mirror html2html traffic to Parsoid/PHP - T229015 T239643 (duration: 13m 29s)

Mentioned in SAL (#wikimedia-operations) [2019-12-18T17:18:02Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@c9d8ef1]: Parsoid-PHP: mirror 100% of all traffic T229015

Mentioned in SAL (#wikimedia-operations) [2019-12-18T17:33:59Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@c9d8ef1]: Parsoid-PHP: mirror 100% of all traffic T229015 (duration: 16m 00s)

Mentioned in SAL (#wikimedia-operations) [2019-12-18T20:36:47Z] <ppchelko@deploy1001> Started deploy [restbase/deploy@6e24349]: Disable all parsoid-php vs parsoid-js special cases T229015

Mentioned in SAL (#wikimedia-operations) [2019-12-18T20:50:43Z] <ppchelko@deploy1001> Finished deploy [restbase/deploy@6e24349]: Disable all parsoid-php vs parsoid-js special cases T229015 (duration: 13m 56s)

ssastry closed this task as Resolved.Dec 20 2019, 7:50 AM
ssastry claimed this task.
ssastry updated the task description. (Show Details)

This is now all done. The unresolved subtasks can be handled independently and should be updated to remove this as the parent task.