Page MenuHomePhabricator

We need a way to do het-deploy in beta-labs for Parsoid/VisualEditor testing
Closed, InvalidPublic

Description

Currently en.wikipedia.beta.wmflabs.org runs master of VisualEditor and a manually-synced version of Parsoid that is the same as production. This is great for testing VE as to whether it's good to deploy into production, and had served us reasonably well so far.

We can't just run Parsoid as master, because (a) Parsoid master is occasionally not compatible with VisualEditor master for days or even weeks at a time as the APIs change, and (b) unless Parsoid is going to be pushed live at the same time as VisualEditor (and split into a het-deploy situation for the cluster's wmfX vs. wmfX+1 deployment) we wouldn't know if VE and Parsoid were truly OK to go out.

However, bug 56583 showed that we also need to test master (and ideally also current production) of VisualEditor against *master* of Parsoid, in Labs, for automated browser testing and so on before any code goes live.

Do we need en.wikipedia.beta.wmflabs.org to have en-Parsoid-master-VE-master.wikipedia.beta.wmflabs.org and en-Parsoid-master-VE-prod.wikipedia.beta.wmflabs.org duplicates? (Eww.)

Also, are there other teams who need their multiple independent components and cross-dependencies? Wikidata, perhaps?


Version: unspecified
Severity: normal

Details

Reference
bz56622

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 2:40 AM
bzimport set Reference to bz56622.
bzimport added a subscriber: Unknown Object (MLST).

In https://gerrit.wikimedia.org/r/#/c/93939/ I made the beta autoupdater to run npm install under common/php-master/extensions/Parsoid/js

Regardless, the parsoid instance is deployment-parsoid2 and runs its own copy of the code under /srv/deployment/parsoid . So it is still being updated manually by whoever has access on it.

The whole point of beta is to test the tips of master branches from all extensions. That let us catch potential issue before all those tips are aggregated in a wmf branch. To prevent new code from being executed, we use feature switching ($wgFeatureXXX = false) that still let us deploy the code and enable it selectively later on.

That being said, I think it would make sense to run VisualEditor and Parsoid on the tip of master. That would probably help you detect possible breakage when deploying.

For the sake of clarification: I am open to any solution :)

  • Bug 56670 has been marked as a duplicate of this bug. ***

Usually Parsoid master will work fine with VE. DOM spec changes have not been frequent in the past, and will get even less frequent over time. Empirically, the main source of issues has been a lack of integration testing rather than existing functionality breaking because of incompatibilities.

So if we could only set up one version for now I'd prioritize testing against master over testing against an older version of Parsoid, in particular if that is not updated automatically. Much of the VE integration testing is also still manual at this point, which makes focusing on the most likely source of errors even more important.

What we could potentially do in Jenkins:

Whenever a change is send on VisualEditor, we could somehow boot up two Parsoid instances running master and production versions then run the VisualEditor tests against each of them.

Similarly, whenever a change is send to Parasoid, we could create an instance of it and run VisualEditor tests from master and production versions.

That should help both team figuring out potential issues and regressions.

We never achieved such integration tests on any project yet. Ideally we would use labs to boot up instances that would contains all the required dependencies and run the tests there. The infrastructure is not ready yet :-/

(In reply to comment #1)

In https://gerrit.wikimedia.org/r/#/c/93939/ I made the beta autoupdater to
run
npm install under common/php-master/extensions/Parsoid/js
Regardless, the parsoid instance is deployment-parsoid2 and runs its own
copy of the code under /srv/deployment/parsoid . So it is still being updated
manually by whoever has access on it.

Copied that to:

Bug 57233 - beta: Parsoid should continuously deploy master branch

I'm not sure I agree this bug is INVALID; we still can't reliably test master of VisualEditor against production of Parsoid or vice versa; instead, we test master of VisualEditor against master of Parsoid in Beta, and production VisualEditor against production Parsoid in production, and hope that we don't get into one of those situations again where one assumes things about the other…