Page MenuHomePhabricator

Fix various dump scripts to not use hardcoded paths to MWScript.php
Closed, ResolvedPublic

Description

Until now we've always relied on a fixed directory layout as seen in production. That means it is really painful to move these scripts to testing platforms where we want to use multiversion processing but have a different layout.

Scripts in question:

  • dumptextPass.php (MW core)
  • utils.py (xml/sql dumps python scripts)
  • dumpcategoriesrdf-shared.sh, dumpcirrussearch.sh, dumpcontentxlation.sh, dump-global-blocks.sh, wikidatadumps-shared.sh (snapshot module in puppet)

Event Timeline

ArielGlenn triaged this task as Medium priority.Sep 20 2018, 2:53 PM
ArielGlenn created this task.

Change 461645 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[mediawiki/core@master] for two-stage dumps make the location of MWScript.php alterable

https://gerrit.wikimedia.org/r/461645

Change 461650 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] make path to MWScript.php configurable for xml/sql dumps

https://gerrit.wikimedia.org/r/461650

Change 461651 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/dumps@master] make location of MWScript.php configurable for xml/sql dumps

https://gerrit.wikimedia.org/r/461651

Change 461667 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] make 'misc cron dumps' use a configured path to MWScript.php

https://gerrit.wikimedia.org/r/461667

Change 461650 merged by ArielGlenn:
[operations/puppet@production] make path to MWScript.php configurable for xml/sql dumps

https://gerrit.wikimedia.org/r/461650

Change 461651 merged by ArielGlenn:
[operations/dumps@master] make location of MWScript.php configurable for xml/sql dumps

https://gerrit.wikimedia.org/r/461651

Change 461667 merged by ArielGlenn:
[operations/puppet@production] make 'misc cron dumps' use a configured path to MWScript.php

https://gerrit.wikimedia.org/r/461667

Two of the symlink fixes have been deployed, one to go.

Change 461645 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[mediawiki/core@master] for two-stage dumps make the location of MWScript.php alterable

https://gerrit.wikimedia.org/r/461645

The last patch is tested and ready to go as soon as there's sign off from a deployer.

Change 461645 merged by jenkins-bot:
[mediawiki/core@master] for two-stage dumps make the location of MWScript.php alterable

https://gerrit.wikimedia.org/r/461645

ArielGlenn updated the task description. (Show Details)

Done. Closing.