Page MenuHomePhabricator

Install pnmtojpeg and ddjvu on UbuntuTrusty contintLabsSlave hosts.
Closed, ResolvedPublic

Description

I tried to move the parsoidsvc-php-parsertests tests over to UbuntuTrusty (node 0.10) in https://gerrit.wikimedia.org/r/174031

This is the result: https://integration.wikimedia.org/ci/job/parsoidsvc-php-parsertests/3043/console

The PHP parser tests failed because various image-processing tools weren't installed on the contintLabsSlave host.

Event Timeline

cscott created this task.Dec 3 2014, 8:45 PM
cscott assigned this task to hashar.
cscott raised the priority of this task from to Needs Triage.
cscott updated the task description. (Show Details)
cscott changed Security from none to None.
cscott added a subscriber: cscott.

The Jenkins slaves have the puppet class mediawik::packages applied and that provides ddjvu.

I confirmed on both Precise and Trusty slaves that:

  • ddjvu is present.
  • pnmtojpeg is not

So it is unlikely it ever worked!

In tests/testHelpers.inc there is a DjVuSupport class which is meant to detect whether all the $wgDjvu* settings are properly set. The class is used when flagging a parser test .txt test with the option 'djvu', which would skip the parser test whenever the installation lacks support ( see tests/parser/parserTest.inc and grep djVuSupport).

So my assumption is that whatever change triggered the build is also introducing a parser tests that require djvu but lacks the 'djvu' option. The job should fail on Precise slave as well.

Triggering the job against a dummy commit on a Trusty instance should pass.

cscott added a subscriber: hashar.EditedDec 3 2014, 11:00 PM

@hashar: nope, see the recheck on patch 11 on https://gerrit.wikimedia.org/r/170455 -- it works when the job was hasSlaveScripts && UbuntuPrecise (the original state, and PS4 -- after the recheck) but fails when the job is contintLabsSlave && UbuntuTrusty (PS3 of https://gerrit.wikimedia.org/r/174031 -- before the recheck). When the job was hasSlaveScripts && UbuntuTrusty (PS2) the job hung in the queued state forever in zuul -- so I guess there's no machine matching that descriptor?

The labels hasContintPackages and hasSlaveScripts are only applied on the production slaves which are both Precise. They are legacy labels which I should most probably cleanup. As a summary:

  • hasSlaveScripts && UbuntuPrecise matches the production slaves (Precise).
  • contintLabsSlave && UbuntuTrusty matches the labs Trusty slaves.
  • hasSlaveScripts && UbuntuTrusty indeed does not match any host and would cause the job to be stuck.

The jobs using Zend php should use UbuntuPrecise which provides PHP 5.3.x.

I have found out that "pnmtojpeg" is only available on the production slaves. The utility is provided by the puppet chain:

  • Package netpbm
  • mediawiki::packages::multimedia
  • mediawiki::multimedia
  • role::mediawiki::imagescaler or role::mediawiki::videoscaler

The contint packages are defined in contint::packages which only includes mediawiki::packages. So I am going to add mediawiki::multimedia there and that will add pnmtojpeg on the contint slaves.

Change 177770 had a related patch set uploaded (by Hashar):
contint: mediawik::multimedia on labs slaves

https://gerrit.wikimedia.org/r/177770

Patch-For-Review

https://gerrit.wikimedia.org/r/177770 has been applied on the contint puppetmaster.

So you can get the job to run on Precise labs slaves using: contintLabsSlave && UbuntuPrecise

hashar triaged this task as Normal priority.Dec 5 2014, 11:07 AM

Change 177770 merged by Alexandros Kosiaris:
contint: mw multimedia packages on labs slaves

https://gerrit.wikimedia.org/r/177770

hashar closed this task as Resolved.Dec 5 2014, 6:38 PM

I think the issue (missing pnmtojpeg on labs slave) is fixed.

The parsoid parsertest job can be migrated to Precise labs instance with contintLabsSlave && UbuntuPrecise (UbuntuTrusty slaves have Zend PHP 5.5.9).

Cool, thanks! I created T76881 to remind me to migrate the last jobs over to node 0.10 now.

hashar mentioned this in Unknown Object (Diffusion Commit).Dec 10 2014, 8:37 PM