FWIW, this would help offline server distributions like the XSCE and many others.

In T117095#1768225, @Metanish wrote:

FWIW, this would help offline server distributions like the XSCE and many others.

@Metanish, are you involved with XSCE? Do you know how they grab the generated files in question? Is it rsync from us or http://kiwix.org/ or none of the above?

Yes. I am one of the volunteers with the project. I believe we grab them off of the kiwix.org download page through a web interface.

More typically though, people keep copies of zim files with them (again downloaded from kiwix.org) and as per requirement, roll a custom set of zim files as per need. To get a sense, please see:
https://docs.google.com/spreadsheets/d/13SA6Grzx70svt_8B4epxaDtbtElg0VlmdcXXg-fth5I/edit#gid=0

@yuvipanda or @chasemp: Is labs in a state to support bare metal deployments in eqiad at this time? I wasn't under the impression it was yet the case. (I thought it was pending the codfw test labs deployment to test labs on bare metal.)

If it isn't ready, can we link in any blocking tasks so its more evident, since this is sitting in the hardware-requests project without all the relevant information.

If it is, then I'll claim this task and start the process of allocations.

Thanks!

This box is to work out the alternative hw case outside of openstack if ironic doesn't pan out

@yuvipanda

How does the implementation of this task looks like?
This need/request is always really valid.

I just spoke to Kelson about this, and I'm willing to set this up if we can provide him with the hardware. Adding a second bare-metal server will be an 'interesting' test case, and Kelson understands the work and responsibilities that will be needed to maintain the box.

promethium has since been allocated elsewhere (to the wikitextexp project), right? Or was that for this purpose?

Correct, the parsing folks are using Promethium. So we would need them to release it, or to rack an additional misc server here.

hang on, I'm confused on why we are pursuing this. The last public statement was we are not going to get entrenched in the hack of hardware in the instance subnet. I would like to have a meeting as to what we are doing and why before we allocate resources. @RobH can you table this until further comment?

OK, sorry -- I may have overpromised here. Can we get a clear description of what this is for and what specifically is insufficient about labs VMs for the solution? Every time we go down this road we seem to conclude that 'bare metal' is a red herring and doesn't actually get us much.

@Andrew @RobH @chasemp @AlexMonk-WMF Thank you for taking time to answer to this ticket. I'm now back on this topic after a long summer pause.

Purpose:
Create ZIM files of all (big) Wikimedia projects. ZIM files are the only solution to have an easy usable offline copy of our projects.

Current situation:
We have been doing these files for years on different systems (meanwhile mostly on labs, except for the few biggest projects)

Problem:
We can not really do monthly snapshot (what would a good thing) and we have pretty serious difficulties to create ZIM files of the biggest Wikipedias (EN, DE, FR). The reason are the limitation in hardware resources. The problem is that we have a quota which blocks us to request one or two more large VMs from the XL/large setup (and there is still a doubt that even the large VM will be able to create a ZIM of WPEN within a month due to CPU and/or storage limitation).

In T117095#2775967, @Kelson wrote:

We can not really do monthly snapshot (what would a good thing) and we have pretty serious difficulties to create ZIM files of the biggest Wikipedias (EN, DE, FR). The reason are the limitation in hardware resources. The problem is that we have a quota which blocks us to request one or two more large VMs from the XL/large setup (and there is still a doubt that even the large VM will be able to create a ZIM of WPEN within a month due to CPU and/or storage limitation).

So you just need a quota bump and perhaps a custom flavor?

@AlexMonk Yes, this is highly probable that we could fix the problem that way. I had created a while ago this request in an attempt to push things this direction #T91976.

A pragmatic approach might be:

• One additional XL VM for DE/FR.
• One additional XL VM for EN (but with a bit more storage, something like 300Go).

By doing this, I think we will have enough HW to make all of the projects one time a month.

@Kelson we did a capacity audit and spoke today about what is possible. We are going to create a custom flavor for the wmoffliner project that is an XL VM with an additional 300G, and allocate that quota along with enough for another XL VM. Hopefully this gets you rolling? At some point down the line we'll end up talking about breaking this out into its own hardware possibly (depending on load and capacity), but we can fulfill this now and so want to :) Thanks for your patience. We will try to get this done soon, the coming holiday (US) may put a bit of delay on it.

@chasemp @Andrew You are awesome! It looks like you just have fixed a many years old HW bootleneck problem! Thank you very much!