Page MenuHomePhabricator

Update mwoffliner for WMF client-specific needs
Closed, InvalidPublic

Description

An umbrella task to track the various changes we may want to make to the mwoffliner logic specifically for the Android app.

Let's discuss these changes with Kiwix to determine how they could best be structured, with the goal of making them generally useful and upstreamable.

Event Timeline

Mholloway renamed this task from Update mwoffliner for client-specific needs to Update mwoffliner for WMF client-specific needs.
Mholloway renamed this task from Update mwoffliner for WMF client-specific needs to Update mwoffliner logic for WMF client-specific needs.Aug 7 2017, 11:44 PM

Why creating a fork? Please collaborate and make PRs. This is how opensource works?

Agreed that a Wikimedia fork would only be desirable to the extent the WMF in-house clients have needs that are of no general interest upstream.

Please do not assume this for any feature, let us discuss this together before if possible.

We won't make any such assumptions! The only changes I've made in my personal fork are a couple of tweaks to get it working correctly in Jessie (including the RangeError fix I upstreamed), and I also patched in in some temporary, ad-hoc timing code so we could get a sense of the range of running times we're looking at (for discussions with ops). We'll definitely discuss planned changes with you and upstream them where they're useful!

By the way, the other change I needed to make to get things working correctly in Jessie was to remove the --strip option from the pngquant call here, as it's not available in the pngquant version in the official Debian Jessie repo and was throwing errors and causing png optimizing to fail. Would you like a PR for that? What's your general philosophy on distro/version compatibility?

Aside from tweaks for running on not-quite-cutting-edge software, there are also some restrictions on where we can pull software from for running in WMF production. It's much more restricted than Cloud VPS/Labs. I think, roughly speaking, software running in WMF prod must come from either the official Ubuntu/Debian repos (almost certainly Debian in this case, as we're phasing out Trusty) or from repos we maintain in-house, so a fork might be needed for that reason (even if identical to upstream or nearly so). (Speaking of which, it looks like openzim software development has moved to Github since this past April—is https://github.com/wikimedia/openzim deprecated?) But I'm not in ops and so I'm not well versed in the details on where we may pull software for running in production, so I'm not sure a fork is needed for this reason.

I really created this task as more of a placeholder for discussion of desired changes.

Mholloway renamed this task from Update mwoffliner logic for WMF client-specific needs to Update mwoffliner for WMF client-specific needs.Aug 12 2017, 2:53 PM
Mholloway updated the task description. (Show Details)
Mholloway updated the task description. (Show Details)
Mholloway updated the task description. (Show Details)
Mholloway updated the task description. (Show Details)