Labs Project Tested: mwoffliner / not applicable
Number of systems: 1
Service: labs on real hardware / mwoffliner
Networking Requirements: TBD
Processor Requirements: 8 Core Xeon
Disks: at least 500G
NIC(s): Doesn't matter
Partitioning Scheme: big /srv
Labs Project Tested: mwoffliner / not applicable
|Declined||• chasemp||T117081 Investigate moving mwoffliner onto a labs-on-real-hardware machine|
|Resolved||• chasemp||T117095 eqiad: 1 hardware access request for labs on real hardware (mwoffliner)|
|Declined||None||T116607 How to handle mgmt lan for labs bare metal?|
Yes. I am one of the volunteers with the project. I believe we grab them off of the kiwix.org download page through a web interface.
More typically though, people keep copies of zim files with them (again downloaded from kiwix.org) and as per requirement, roll a custom set of zim files as per need. To get a sense, please see:
@yuvipanda or @chasemp: Is labs in a state to support bare metal deployments in eqiad at this time? I wasn't under the impression it was yet the case. (I thought it was pending the codfw test labs deployment to test labs on bare metal.)
If it isn't ready, can we link in any blocking tasks so its more evident, since this is sitting in the hardware-requests project without all the relevant information.
If it is, then I'll claim this task and start the process of allocations.
I only seem to have a single spare that meets these requirements closely:
Dell PowerEdge R420, Dual Intel Xeon E5-2440, 32GB Memory, Dual 300GB SSD, Dual 500GB Nearline SAS
promethium (warranty expired in early 2015)
Then we have a one other that is overkill on disk space:
Dell PowerEdge R420, dual Intel Xeon E5-2450 v2 2.50GHz, 64GB Memory, (4) 3TB Disks
wmf4541 (warranty expires on 2017-03-19)
So Promethium seems to be the best choice for allocation. I'm assigning this to @mark for his approval, as its allocating an out of warranty spare (though one of the newer out of warranty spares) into further use. (This isn't as bad as most of the eqiad spares, as they are well of our warranty but far less memory and unsuitable for this request.)
@mark: Please comment relevant approvals/questions/clarifications and assign back to either me for implementation or to Yuvi for followup questions.
I just spoke to Kelson about this, and I'm willing to set this up if we can provide him with the hardware. Adding a second bare-metal server will be an 'interesting' test case, and Kelson understands the work and responsibilities that will be needed to maintain the box.
hang on, I'm confused on why we are pursuing this. The last public statement was we are not going to get entrenched in the hack of hardware in the instance subnet. I would like to have a meeting as to what we are doing and why before we allocate resources. @RobH can you table this until further comment?
OK, sorry -- I may have overpromised here. Can we get a clear description of what this is for and what specifically is insufficient about labs VMs for the solution? Every time we go down this road we seem to conclude that 'bare metal' is a red herring and doesn't actually get us much.
Create ZIM files of all (big) Wikimedia projects. ZIM files are the only solution to have an easy usable offline copy of our projects.
We have been doing these files for years on different systems (meanwhile mostly on labs, except for the few biggest projects)
We can not really do monthly snapshot (what would a good thing) and we have pretty serious difficulties to create ZIM files of the biggest Wikipedias (EN, DE, FR). The reason are the limitation in hardware resources. The problem is that we have a quota which blocks us to request one or two more large VMs from the XL/large setup (and there is still a doubt that even the large VM will be able to create a ZIM of WPEN within a month due to CPU and/or storage limitation).
@AlexMonk Yes, this is highly probable that we could fix the problem that way. I had created a while ago this request in an attempt to push things this direction #T91976.
A pragmatic approach might be:
- One additional XL VM for DE/FR.
- One additional XL VM for EN (but with a bit more storage, something like 300Go).
By doing this, I think we will have enough HW to make all of the projects one time a month.
@Kelson we did a capacity audit and spoke today about what is possible. We are going to create a custom flavor for the wmoffliner project that is an XL VM with an additional 300G, and allocate that quota along with enough for another XL VM. Hopefully this gets you rolling? At some point down the line we'll end up talking about breaking this out into its own hardware possibly (depending on load and capacity), but we can fulfill this now and so want to :) Thanks for your patience. We will try to get this done soon, the coming holiday (US) may put a bit of delay on it.