Change Details

When we stop treating Parsoid/PHP as an extension, we need a deployment strategy that transitions over requests across the deployment boundary without breaking anything. We also need this to work with any necessary reverts. ---- # Plan TL;DR: all machines (in main and parsoid cluster) are identical. Parsoid has its own ‘vendor’ branch checked out in a special place, and startup configuration switches parsoid cluster machines to use this. Every deploy (of both Parsoid and the train) touches all machines in both main and parsoid cluster, so we’re always in sync. 1. Parsing team code lands in parsoid.git master branch, as usual. 1. Prior to deploy, parsing team manually updates scandium w/ the version of parsoid they hope to deploy and triggers a test run (round-trip testing script). Scandium has a persistent symlink or some other magic to enable it to run parsoid from a git checkout. TBD exactly how this works, but scandium will be an evil hack. 2. If that looks good, we tag that version in the parsoid repo as `0.12.0-a<N>` and push the tag (`git push origin 0.12.0-a<N>`) 3. Checkout `mediawiki/vendor.git` *parsoid* branch. 1. *First* reset the parsoid branch to match master ([from SO](https://stackoverflow.com/questions/4911794/git-command-for-making-one-branch-like-another/4912267#4912267)): `$ git fetch origin` `$ git checkout -b tmp origin/master` `$ git merge -s ours origin/parsoid` `$ git checkout parsoid` `$ git reset --hard tmp` `$ git branch -D tmp` 2. Then update composer.json to reference parsoid 0.12.0-a<N>, then do `composer update --no-dev` (which should only update parsoid) 3. git add mediawiki/parsoid composer.lock composer.json ; git commit -m "deploy parsoid 0.12.0-a<N> to parsoid cluster" ; git review parsoid 4. Review & merge. May or may not go live on beta right away, depends on it James F hacks beta to hook this up properly. *Beta deploy process TBD* Beta cluster ?auto? updates its checkout of /srv/mediawiki/parsoid-vendor. 5. *Window of danger*: next scap (whether by parsoid team or releng for an UBN) will deploy this to all parsoid and main cluster machines. 1. If something goes wrong, roll back the scap first (takes ~75s) then revert gerrit patch and merge revert. 6. This new version of parsoid does *not* automatically ride the train; have to do a separate commit to `master` branch for that. 1. In the future as we gain confidence, the workflow might shift to committing the new parsoid version to master first and then "merge theirs" it over to the parsoid branch, so that the new version of parsoid rides the next train w/o any special effort. 7. From deploy1001, a Parsing team member does `scap sync-dir /srv/mediawiki/parsoid-vendor Updating Parsoid to XXX version` ** This **makes the new version of Parsoid live in production for VRS requests, //i.e.// VisualEditor** 8. If all is well, hurrah; if things go bad, they `git revert` in /srv/mediawiki/parsoid-vendor and re-sync. 9. From time to time, Parsing team write a patch updating vendor.git master branch with a newer version of Parsoid, which then rolls out in the train. ** This **makes the new version of Parsoid live in production for non-VRS requests, //e.g.// interface parse requests**

When we stop treating Parsoid/PHP as an extension, we need a deployment strategy that transitions over requests across the deployment boundary without breaking anything. We also need this to work with any necessary reverts. ---- # Plan TL;DR: all machines (in main and parsoid cluster) are identical. Parsoid has its own ‘vendor’ branch checked out in a special place, and startup configuration switches parsoid cluster machines to use this. Every deploy (of both Parsoid and the train) touches all machines in both main and parsoid cluster, so we’re always in sync. 1. Parsing team code lands in parsoid.git master branch, as usual. 1. Prior to deploy, parsing team manually updates scandium w/ the version of parsoid they hope to deploy and triggers a test run (round-trip testing script). Scandium has a persistent symlink or some other magic to enable it to run parsoid from a git checkout. TBD exactly how this works, but scandium will be an evil hack. 2. If that looks good, we tag that version in the parsoid repo as `0.12.0-a<N>` and push the tag (`git push origin 0.12.0-a<N>`) 3. Checkout `mediawiki/vendor.git` *parsoid* branch. 1. *First* reset the parsoid branch to match master ([from SO](https://stackoverflow.com/questions/4911794/git-command-for-making-one-branch-like-another/4912267#4912267)): `$ git fetch origin` `$ git checkout -b tmp origin/master` `$ git merge -s ours origin/parsoid` `$ git checkout parsoid` `$ git reset --hard tmp` `$ git branch -D tmp` 2. Then update composer.json to reference parsoid 0.12.0-a<N>, then do `composer update --no-dev` (which should only update parsoid) 3. git add mediawiki/parsoid composer.lock composer.json ; git commit -m "deploy parsoid 0.12.0-a<N> to parsoid cluster" ; git review parsoid 4. Review & merge. May or may not go live on beta right away, depends on it James F hacks beta to hook this up properly. *Beta deploy process TBD* Beta cluster ?auto? updates its checkout of /srv/mediawiki/parsoid-vendor. 5. *Window of danger*: next scap (whether by parsoid team or releng for an UBN) will deploy this to all parsoid and main cluster machines. 1. If something goes wrong, roll back the scap first (takes ~75s) then revert gerrit patch and merge revert. 6. This new version of parsoid does *not* automatically ride the train; have to do a separate commit to `master` branch for that. 1. In the future as we gain confidence, the workflow might shift to committing the new parsoid version to master first and then "merge theirs" it over to the parsoid branch, so that the new version of parsoid rides the next train w/o any special effort. 7. From deploy1001, a Parsing team member does `scap sync-dir /srv/mediawiki/parsoid-vendor 'Updating Parsoid to XXX version'` ** This **makes the new version of Parsoid live in production for VRS requests, //i.e.// VisualEditor** 8. If all is well, hurrah; if things go bad, they `git revert` in /srv/mediawiki/parsoid-vendor and re-sync. 9. From time to time, Parsing team write a patch updating vendor.git master branch with a newer version of Parsoid, which then rolls out in the train. ** This **makes the new version of Parsoid live in production for non-VRS requests, //e.g.// interface parse requests**

When we stop treating Parsoid/PHP as an extension, we need a deployment strategy that transitions over requests across the deployment boundary without breaking anything. We also need this to work with any necessary reverts. ---- # Plan TL;DR: all machines (in main and parsoid cluster) are identical. Parsoid has its own ‘vendor’ branch checked out in a special place, and startup configuration switches parsoid cluster machines to use this. Every deploy (of both Parsoid and the train) touches all machines in both main and parsoid cluster, so we’re always in sync. 1. Parsing team code lands in parsoid.git master branch, as usual. 1. Prior to deploy, parsing team manually updates scandium w/ the version of parsoid they hope to deploy and triggers a test run (round-trip testing script). Scandium has a persistent symlink or some other magic to enable it to run parsoid from a git checkout. TBD exactly how this works, but scandium will be an evil hack. 2. If that looks good, we tag that version in the parsoid repo as `0.12.0-a<N>` and push the tag (`git push origin 0.12.0-a<N>`) 3. Checkout `mediawiki/vendor.git` *parsoid* branch. 1. *First* reset the parsoid branch to match master ([from SO](https://stackoverflow.com/questions/4911794/git-command-for-making-one-branch-like-another/4912267#4912267)): `$ git fetch origin` `$ git checkout -b tmp origin/master` `$ git merge -s ours origin/parsoid` `$ git checkout parsoid` `$ git reset --hard tmp` `$ git branch -D tmp` 2. Then update composer.json to reference parsoid 0.12.0-a<N>, then do `composer update --no-dev` (which should only update parsoid) 3. git add mediawiki/parsoid composer.lock composer.json ; git commit -m "deploy parsoid 0.12.0-a<N> to parsoid cluster" ; git review parsoid 4. Review & merge. May or may not go live on beta right away, depends on it James F hacks beta to hook this up properly. *Beta deploy process TBD* Beta cluster ?auto? updates its checkout of /srv/mediawiki/parsoid-vendor. 5. *Window of danger*: next scap (whether by parsoid team or releng for an UBN) will deploy this to all parsoid and main cluster machines. 1. If something goes wrong, roll back the scap first (takes ~75s) then revert gerrit patch and merge revert. 6. This new version of parsoid does *not* automatically ride the train; have to do a separate commit to `master` branch for that. 1. In the future as we gain confidence, the workflow might shift to committing the new parsoid version to master first and then "merge theirs" it over to the parsoid branch, so that the new version of parsoid rides the next train w/o any special effort. 7. From deploy1001, a Parsing team member does `scap sync-dir /srv/mediawiki/parsoid-vendor 'Updating Parsoid to XXX version`'` ** This **makes the new version of Parsoid live in production for VRS requests, //i.e.// VisualEditor** 8. If all is well, hurrah; if things go bad, they `git revert` in /srv/mediawiki/parsoid-vendor and re-sync. 9. From time to time, Parsing team write a patch updating vendor.git master branch with a newer version of Parsoid, which then rolls out in the train. ** This **makes the new version of Parsoid live in production for non-VRS requests, //e.g.// interface parse requests**