Page MenuHomePhabricator

Finish productionizing/puppetizing the mwoffliner zim creation environment
Closed, InvalidPublic

Description

A lot of the dependency installation and configuration is set up in the new zim_builder module currently living in my home directory on zim-proto.mobile.eqiad.wmflabs. Remaining work to do:

  • Puppetize the necessary redis config changes (done manually on zim-proto)
  • Puppetize the actual mwoffliner install (done with a manual npm install on zim-proto)
  • Get all of this running more smoothly / improve the order of operations
  • Figure out what to do about libgumbo-dev
  • w/r/t libgumbo-dev, one workaround to consider is using zimmer, a pure-nodejs drop-in replacement for zimwriterfs that doesn't require libgumbo. The tradeoff is that zimmer doesn't support full-text search indexing.

Event Timeline

Mholloway created this task.Aug 8 2017, 1:18 AM
Mholloway updated the task description. (Show Details)Aug 8 2017, 2:17 AM
Kelson added a subscriber: Kelson.Aug 9 2017, 8:11 PM

What is the problem with libgumbo-dev? Would not be an option to adopt and help to finish quickly a first version of https://github.com/openzim/zimfarm?

The issue with libgumbo-dev is that it isn't packaged for Jessie, which is what we'd almost certainly be running on in production.

I've worked around it in Cloud VPS by just building and installing gumbo-parser from source.

Kelson added a comment.Aug 9 2017, 9:00 PM

@Mholloway OK :( It's available on recent versions of Ubuntu, so it is also probably for Debian.

Mholloway closed this task as Invalid.Apr 12 2018, 2:46 PM

This work is stalled and the circumstances leading to this ticket will likely have changed completely if and when this work gets picked up again.