Right now it's assumed that everything running on these snapshots writes to a single nfs-mounted file system. That's how it's always been. In order to move the xml/sql dumps to use the dumpsdata host filesystem and not move everything at once, we need to permit two mount points and two paths.
Description
Description
Details
Details
Related Changes in Gerrit:
| Subject | Author | Repo | Branch | Lines +/- | |
|---|---|---|---|---|---|
| use separate path for public/other datasets | ArielGlenn | operations/puppet | production | +32 -22 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | ArielGlenn | T178893 Move xml/sql dumps to dumpsdata1001 from dataset1001 | |||
| Resolved | ArielGlenn | T178888 make snapshot cron jobs use different configs and mount path than xml/sql dumps |
Event Timeline
Comment Actions
All cron jobs but the wikidata dumps write somewhere on /mnt/data/xmldatadumps/public/other/... except I think for the wikidata cron jobs (@hoo ?) so these should write there with whatever symlink changes need to be made. I know there are already some links for backward compat.
Comment Actions
Change 386161 had a related patch set uploaded (by ArielGlenn; owner: ArielGlenn):
[operations/puppet@production] use separate path for public/other datasets
Comment Actions
Except for the temp dir, the Wikidata entity dumpers write to public/other/wikidata (legacy symlinks) and public/other/wikibase/wikidatawiki (actual data).
Comment Actions
The point is that later sometime in the future it will be (say) /mnt/someotherthing/wikidata and /mnt/someotherthing/wikibase/wikidatawiki while /mnt/data/xmldatadumps/public will be some different thing. No symlinks are made or anything into the directory above (public)?
Comment Actions
Only https://dumps.wikimedia.org/wikidatawiki/entities/ and the symlinks inside of other/ mentioned before.
Comment Actions
Change 386161 merged by ArielGlenn:
[operations/puppet@production] use separate path for public/other datasets
Comment Actions
I shoulda closed this when it got merged. Doing so now, as xml/sql dumps are written on a different file system than the cron job misc dumps, this very minute.