Page MenuHomePhabricator

put static-bugzilla HTML dump on dumps servers
Closed, ResolvedPublic

Description

I want to put a .tar.gz of all the HTML on https://static-bugzilla.wikimedia.org/

and add it next to our existing dump of the Bugzilla DB at:

https://dumps.wikimedia.org/other/bugzilla/

The size is 3.6GB uncompressed and the compressed tarball I want to put on the servers is 440MB.

Amongst the reasons is that then I have a place to pull it from to build a container with it but also of course to make it convenient for users to save a copy of all the data at once.

Ariel mentioned that data needs to be on both labstore1006 and labstore1007 and to talk to Brooke about it before doing that and Brooke mentioned there might currently be disk space issues.

So this is to get back to that some time later.

Event Timeline

The space issues are at T281203 so yeah, that needs to get figured out.

The size I want to add here is 3.6G _uncompressed_ so it should be only a couple hundred MB in a single .tar.gz that I actually want to add. Given that other discussion there talks about Terabytes.. seems like a drop in the ocean though, maybe ?

I copied the data to @deploy1002:/srv/miscweb (using gerrit:699064 and compressed it over there and it's 440MB .tar.gz with default settings.

Hi @ArielGlenn I am trying to get back to this ticket. Remember the space issues you mentioned above in June 2021? I see that ticket is closed as resolved (T281203). But of course I can't ask Brooke now. Any suggestion how to move forward? Could I move 440MB over there now or who should I ask ideally?

I see on one hand that the ticket talks about NOT getting new hardware and declines.. but on the other hand I see 15T free on labstore1006/labstore1007 which makes me think this is not even worth the ping.

Dzahn changed the task status from Open to In Progress.Mar 25 2022, 6:43 PM

Change 773832 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] dumps: add description for Bugzilla HTML dump file

https://gerrit.wikimedia.org/r/773832

Change 773832 merged by Dzahn:

[operations/puppet@production] dumps: add description for Bugzilla HTML dump file

https://gerrit.wikimedia.org/r/773832

Mentioned in SAL (#wikimedia-operations) [2022-03-25T19:10:10Z] <mutante> copying dump from deploy server to dumps server: scp -3 deploy1002.eqiad.wmnet:/srv/miscweb/static-bugzilla.tar.gz labstore1006.wikimedia.org:~ (T284193)

Talked to Andrew Bogott about this on IRC.

Moved the file over from deployment server to dumps server, labstore1006 as bugzilla_static_html_files.tar.gz

https://dumps.wikimedia.org/other/bugzilla/

This is done. Puppet patch to edit HTML descriptiong. Uploaded files to existing /bugzilla/ dir on BOTH labstore1006 and labstore1007. removed the temp files from deploy1002