Page MenuHomePhabricator

Craft a deployment strategy to transition Parsoid/PHP from a faux extension to a composer library without breaking incoming requests
Closed, ResolvedPublic

Description

When we stop treating Parsoid/PHP as an extension, we need a deployment strategy that transitions over requests across the deployment boundary without breaking anything. We also need this to work with any necessary reverts.


The Plan

TL;DR: all machines (in main and parsoid cluster) are identical. Parsoid is loaded out of the regular hetdeploy ‘vendor’ branch. Startup configuration means the "parsoid cluster" machines have some extra config to expose the REST routes. Every deploy (of both Parsoid and the train) touches all machines in both main and parsoid cluster, so we’re always in sync.

  1. Parsing team code lands in parsoid.git master branch, as usual.
  2. Prior to deploy, parsing team manually updates /srv/parsoid-testing on scandium w/ the version of parsoid they hope to deploy (update_parsoid.sh) and triggers tests (see round-trip testing instructions). The mediawiki-config file CommonSettings.php has a special case for scandium which makes it load Parsoid from /srv/parsoid-testing and not the usual vendor repository.
  3. If the scandium test results look good, we tag that version in the parsoid repo as 0.12.0-a<N> and push the tag (git tag 0.12.0-a<N>; git push origin 0.12.0-a<N>)
  4. Checkout mediawiki/vendor.git *master* branch.
    1. update composer.json to reference parsoid 0.12.0-a<N>,
    2. do composer update --no-dev (which should only update parsoid)
    3. git add wikimedia/parsoid composer.lock composer.json <etc>; git commit -m "Upgrade parsoid to 0.12.0-a<N>" ; git review
  5. Review & merge. Will go live on beta cluster right away.
  6. The next time the train runs, it will take the master of vendor.git, including any new version of Parsoid, and deploy it to production, intially only to testwikis, then to group0, then more.
  7. Review impact.
    1. If all is well, hurrah.
    2. if things go bad, the train is pulled back, and if necessary the new version of Parsoid is reverted from, or a new fixed release is pushed in vendor.git and cherry-picked to production.

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
ResolvedJdforrester-WMF
ResolvedJdforrester-WMF
OpenNone
OpenNone
OpenNone
ResolvedGoalcscott
Resolvedcscott
ResolvedDzahn
DeclinedJdforrester-WMF
ResolvedJdforrester-WMF
ResolvedJdforrester-WMF
ResolvedKrenair
ResolvedKrenair
ResolvedDzahn
ResolvedKrenair
Resolvedcscott

Event Timeline

ssastry triaged this task as Medium priority.Dec 6 2019, 9:58 PM
ssastry created this task.

"Just" have the version of MW-core where the library is introduced hard-code the pointer?

ssastry moved this task from Backlog to Deployment on the Parsoid-PHP board.Dec 8 2019, 3:18 AM

Change 572047 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/vendor@master] Add wikimedia/parsoid at 0.12.0-a1

https://gerrit.wikimedia.org/r/572047

Change 572035 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Add Parsoid as a library dependency

https://gerrit.wikimedia.org/r/572035

Change 572035 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/core@master] Add Parsoid as a library dependency

https://gerrit.wikimedia.org/r/572035

Change 572047 merged by jenkins-bot:
[mediawiki/vendor@master] Add wikimedia/parsoid at 0.12.0-a2

https://gerrit.wikimedia.org/r/572047

Change 572035 merged by jenkins-bot:
[mediawiki/core@master] Add Parsoid as a library dependency

https://gerrit.wikimedia.org/r/572035

cscott updated the task description. (Show Details)Feb 21 2020, 10:24 PM
Krinkle updated the task description. (Show Details)Feb 21 2020, 10:32 PM
cscott updated the task description. (Show Details)Feb 21 2020, 10:34 PM
cscott updated the task description. (Show Details)Feb 21 2020, 10:47 PM

Change 575336 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[operations/mediawiki-config@master] Parsoid: Use the version of Parsoid in $IP/vendor

https://gerrit.wikimedia.org/r/575336

bd808 added a subscriber: bd808.Feb 27 2020, 11:46 PM

Change 575336 merged by jenkins-bot:
[operations/mediawiki-config@master] Parsoid: Use the version of Parsoid in $IP/vendor

https://gerrit.wikimedia.org/r/575336

Mentioned in SAL (#wikimedia-operations) [2020-03-03T00:13:05Z] <jforrester@deploy1001> Started scap: wmf-config/CommonSettings.php T240055: Point the Parsoid cluster at the train version of Parsoid, not a special check-out

Mentioned in SAL (#wikimedia-operations) [2020-03-03T00:13:10Z] <jforrester@deploy1001> sync aborted: wmf-config/CommonSettings.php T240055: Point the Parsoid cluster at the train version of Parsoid, not a special check-out (duration: 00m 03s)

Mentioned in SAL (#wikimedia-operations) [2020-03-03T00:14:17Z] <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: T240055: Point the Parsoid cluster at the train version of Parsoid, not a special check-out (duration: 00m 56s)

Mentioned in SAL (#wikimedia-operations) [2020-03-03T00:53:22Z] <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: T240055 Update special scandium configuration to load from /srv/parsoid-testing (duration: 00m 58s)

Provisionally, this now "works".

Change 576979 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[operations/puppet@production] Update scandium (parsoid testing machine) to reflect new Parsoid configuration

https://gerrit.wikimedia.org/r/576979

Change 572034 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[mediawiki/services/parsoid@master] Ensure Parsoid on scandium (RT test server) executes from git checkout

https://gerrit.wikimedia.org/r/572034

Change 576990 had a related patch set uploaded (by C. Scott Ananian; owner: C. Scott Ananian):
[operations/puppet@production] Remove Parsoid node service; replace with git checkout on RT testing server

https://gerrit.wikimedia.org/r/576990

Change 576979 merged by Dzahn:
[operations/puppet@production] Update scandium (parsoid testing machine) to reflect new Parsoid configuration

https://gerrit.wikimedia.org/r/576979

Change 576990 merged by Dzahn:
[operations/puppet@production] Check out parsoid from git on paroid::testing machines

https://gerrit.wikimedia.org/r/576990

Change 572034 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Ensure Parsoid on scandium (RT test server) executes from git checkout

https://gerrit.wikimedia.org/r/572034

ssastry assigned this task to cscott.Mar 6 2020, 5:05 PM
ssastry raised the priority of this task from Medium to High.
cscott updated the task description. (Show Details)Mar 6 2020, 7:20 PM
cscott updated the task description. (Show Details)
cscott updated the task description. (Show Details)Mar 12 2020, 12:29 AM