Page MenuHomePhabricator

K3. Run Flow tests with Parsoid on Jenkins
Open, NormalPublic3 Story Points

Description

@EBernhardson said that they are currently run with wikitext (without Parsoid). It would be good to run them with Parsoid (since that's our main backend) in addition to or instead of wikitext.


Trello card: YpgazY2B

Event Timeline

Spage raised the priority of this task from to Normal.Dec 3 2014, 11:30 AM
matthiasmullie added a subscriber: hashar.

Copying @hashar's comment on Trello:

Sorry for the rather long delay. To run tests with Parsoid as a backend, we would need a job that:

  • spawn a Parsoid daemon like we do for some Parsoid integration tests
  • setup MediaWiki behind a web server which we do for qunit tests

The mwext-Flow-testextension job execute the PHPUnit test suite which combines unit tests for the Flow function as well as integration tests against mediawiki/core.

It is totally doable to spawn a Parsoid daemon and run whatever tests you want. That being said, I would prefer to avoid rushing a solution that would match just the Flow extension in favor of a system that would work for other use cases (such as VisualEditor).

During the summer, I worked on a set of utilities that would let us clone core + vendor + extensions and run them all together with the appropriate patches/branches checked out. That will yield a state of code to be tested and is subject of an RFC: https://www.mediawiki.org/wiki/RFC/Extensions_continuous_integration

Once we have that stack of code available, everything is loaded via include() and we will run: the PHPUnit test suite the qunit page

That will supersede all the testextension and extension-qunit jobs in favor of a two jobs that are used by all repositories.

We can come up with a third job that would have a bit more backend configured such as ElasticSearch or Parsoid. But that will take a bit more time since I want the other two jobs to be setup first.

As I understand, running Flow tests with Parsoid is more of a nice to have feature than a breaker and I guess it can wait a bit. In short: yes, but a bit later and on a wider scale :-)

I am subscribed to this Trello card so we can keep the discussion here 8-)

So i know that doing this "correct" is of big benefit, but could we have most of the benefits now just by setting Flow config to point to already existing parsoid in labs? Without having to install or boot a parsoid instance the changes required become trivial, as long as the jenkins runners have network access to the labs machines.

@hashar: How far of do you think those proposed ideas are?
Would it make sense to contact an existing Parsoid, until we can properly clone & run it in isolation?
We're not in a huge rush, but it would be nice to have our tests run with Parsoid.

Possible implementation suggestion, copied from Trello:

http://www.mediawiki.org/wiki/Parsoid/API says "A production version of Parsoid on the Wikimedia cluster can be accessed at http://parsoid-lb.eqiad.wikimedia.org/", so I guess that one would be good to use.

I'm no Jenkins expert, but I think we should add a build step "Execute shell" with the following command on https://integration.wikimedia.org/ci/view/Extensions/job/mwext-Flow-testextension/configure - this would override the default Parsoid settings:

# Load context (namely: MW_INSTALL_PATH)
. "/srv/deployment/integration/slave-scripts/bin/mw-set-env.sh"

# Configure Parsoid
echo -e \
 "<?php\n\$wgFlowParsoidURL = 'http://parsoid-lb.eqiad.wikimedia.org/';\n"\
 "\$wgFlowParsoidPrefix = 'enwiki';\n"\
 "\$wgFlowParsoidTimeout = 100;\n" >> "$MW_INSTALL_PATH/LocalSettings.php"
Spage edited a custom field.Jan 17 2015, 6:06 AM

I think @EBernhardson changed most/all of these to use a fixture (mock output for given input) if Parsoid is not available. We'd still like the real thing, but that should help.

Mattflaschen-WMF renamed this task from K3. Run Flow unit tests with Parsoid (3) to K3. Run Flow unit tests with Parsoid on Jenkins (3).Mar 26 2015, 2:53 AM
Restricted Application added a project: Growth-Team. · View Herald TranscriptJul 18 2018, 7:01 PM
SBisson moved this task from Inbox to Triaged but Future on the Growth-Team board.Jul 20 2018, 5:53 PM
Krinkle renamed this task from K3. Run Flow unit tests with Parsoid on Jenkins (3) to K3. Run Flow tests with Parsoid on Jenkins.Apr 13 2019, 1:39 AM
Krinkle added a subscriber: EBernhardson.