Page MenuHomePhabricator

DRY up .html files in puppet used for snapshot and dumps modules
Closed, ResolvedPublic

Description

E.g. legal.html exists in both puppet modules with (mostly?) the same content. This can lead to confusion when changes need to be made like happened in T278409.

Event Timeline

Change 678719 had a related patch set uploaded (by ArielGlenn; author: ArielGlenn):

[operations/puppet@production] remove obsolete html files from snapshot manifests for dumps

https://gerrit.wikimedia.org/r/678719

Change 678719 merged by ArielGlenn:

[operations/puppet@production] remove obsolete html files from snapshot manifests for dumps

https://gerrit.wikimedia.org/r/678719

OK, I have had a look at what's going on, and it is amusing, at least to me at the start of a fresh work week.

TL;DR: E_TOOMANYCOPIES, the snapshot copy can go. But some copies on the dumpsdata hosts needed to be manually removed too.

Long version:

The dumps code repo has templates for an index.html file to be written, which contains a link to legal.html; these sample html files are provided for any third party as examples. This presumes that there's one server where xml/sql dumps are written, and so you plop the files in there one time and you're done.

We, however, because nothing is that simple around here, have the following setup:

  • snapshots have the full set of html templates, including legal.html, even though strictly speaking it's not a template. Some are (like the html snippet for all the info for one wiki).
  • the dumpsdata hosts had a July 2018 copy of legal.html, which got on there, well I have no idea how now, via some previous puppet setup.
  • the labstore boxes have the version in the dumps module, but there is an rsync of all top-level html files from the public dumps output dir on one dumpsdata host to the labstores. Strictly speaking it's a periodic rsync of a tarball of these, which is then unpacked.

The upshot was:

  • changing the snapshot version does nothing.
  • changing the labstore version changes something, but only until the next tarball unpack, which puts the 2018 version back...until puppet runs and overwrites it again. Etc.

I've removed the old dumpsdata copies, removed the snapshot manifest copies, and left the labstore copies. That should do it.
I've done the same for dvd.html, which hasn't seen content change since 2015 :-)

@Ottomata I think this is good to go; care to double-check that things look sane from your side?

Thanks, see you in the channels!