Page MenuHomePhabricator

Update roundtrip-test.js script to pick one of Parsoid/JS or Parsoid/PHP + pass the right headers & url for Parsoid/PHP + disable git HEAD based test runs
Closed, ResolvedPublic0 Estimated Story Points

Description

T229363: Write and merge puppet patch to update parsoid-rt-client.config.js to talk to Parsoid/PHP adds a parsoidPHPURL to the client config.

We need two things:

  1. Now, we need a mechanism for roundtrip-test.js to use the current default (JS) url or pick the PHP url so we run rt testing with Parsoid/JS or Parsoid/PHP. One way to do this is for the script to check for the presence of a file (or its contents) on disk that we manually toggle to pick what version we want to run. This lets us switch versions on our own without code commits or puppet changes.
  1. The Parsoid/JS curl command is curl http://localhost:8142/<domain>/v3/page/html/<title>/<revid> and Parsoid/PHP curl command is curl -x scandium.eqiad.wmnet:80 http://<domain>/w/rest.php/<domain>/v3/page/html/<title>/<revid>. So, we need to ensure we compute the parsoid/php url appropriately. It will not be based on parsoidPHPURL as it is set up in puppet. We could make it a format-string instead. Plus, we need to direct the request to the proxy, not localhost. So, we may need to enhance the client settings in puppet to include that as well.
  1. parsoid-rt test runs are automatically triggered based on changes to git HEAD in /srv/deployment/parsoid/deploy/src. This setting is in /etc/testreduce/parsoid-rt-client.config.js (which comes from puppet) in the gitRepoPath property. However, testreduce service has another option to trigger new tests. You can provide it a gitCommitFetch property which is a function that testreduce clients run to figure out the test run id. Take a look at /etc/testreduce/parsoid-vd-client.config.js. We will probably want to use that same mechanism to manually trigger new test runs. This lets us control when to use scandium for running roundtrip tests and when to use scandium for performance testing. So, we can continually update code on scandium when bugs are fixed and run performance tests without triggering parser tests and heavily loading the server as a result.

Event Timeline

ssastry triaged this task as High priority.Aug 8 2019, 8:35 PM
ssastry created this task.

Change 529191 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[mediawiki/services/parsoid@master] RT Testing: Pick Parsoid/JS or Parsoid/PHP based on presence of a file

https://gerrit.wikimedia.org/r/529191

ssastry renamed this task from Update roundtrip-test.js script to pick one of Parsoid/JS or Parsoid/PHP based on some filesystem state we toggle to Update roundtrip-test.js script to pick one of Parsoid/JS or Parsoid/PHP + pass the right headers & url for Parsoid/PHP.Aug 9 2019, 2:07 PM
ssastry updated the task description. (Show Details)
ssastry added a subscriber: Dzahn.
ssastry renamed this task from Update roundtrip-test.js script to pick one of Parsoid/JS or Parsoid/PHP + pass the right headers & url for Parsoid/PHP to Update roundtrip-test.js script to pick one of Parsoid/JS or Parsoid/PHP + pass the right headers & url for Parsoid/PHP + disable git HEAD based test runs.Aug 9 2019, 2:15 PM
ssastry updated the task description. (Show Details)
ssastry added a subscriber: Arlolra.

@Arlolra This is something one of us should tackle including changes to the puppet file. This doesn't have to be today.

While this is fresh in my mind, I'll take a whack at it.

Change 529391 had a related patch set uploaded (by Subramanya Sastry; owner: Subramanya Sastry):
[operations/puppet@production] Update parsoid-rt-client.config.js.erb to fetch test ids from a function

https://gerrit.wikimedia.org/r/529391

Change 529191 merged by jenkins-bot:
[mediawiki/services/parsoid@master] RT testing: Get test ids from a file; 'PHP:' prefix triggers Parsoid/PHP

https://gerrit.wikimedia.org/r/529191

Change 529391 merged by Giuseppe Lavagetto:
[operations/puppet@production] Update parsoid-rt-client.config.js.erb to fetch test ids from a function

https://gerrit.wikimedia.org/r/529391

Change 531556 had a related patch set uploaded (by Arlolra; owner: Subramanya Sastry):
[mediawiki/services/parsoid@master] roundtrip-test.js: Support Parsoid/PHP server options

https://gerrit.wikimedia.org/r/531556

Change 531556 merged by jenkins-bot:
[mediawiki/services/parsoid@master] roundtrip-test.js: Support Parsoid/PHP server options

https://gerrit.wikimedia.org/r/531556

Change 532331 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[operations/puppet@production] Scandium: Add the protocol to the rt-client config

https://gerrit.wikimedia.org/r/532331

Change 532331 merged by Giuseppe Lavagetto:
[operations/puppet@production] Scandium: Add the protocol to the rt-client config

https://gerrit.wikimedia.org/r/532331