CI job fails with: /usr/lib/x86_64-linux-gnu/hhvm/extensions/20150212/tidy.so: cannot open shared object file: No such file or directory
Closed, ResolvedPublic

Description

https://integration.wikimedia.org/ci/job/mediawiki-extensions-hhvm-jessie/16610/console

+ /srv/deployment/integration/slave-scripts/bin/mw-install-mysql.sh
00:01:16.182 [Tue Jun 27 15:16:37 2017] [hphp] [2066:7fb901ddb200:0:000002] [] Uncaught exception: Could not open extension /usr/lib/x86_64-linux-gnu/hhvm/extensions/20150212/tidy.so: /usr/lib/x86_64-linux-gnu/hhvm/extensions/20150212/tidy.so: cannot open shared object file: No such file or directory
hashar created this task.Jun 27 2017, 6:04 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 27 2017, 6:04 PM
hashar claimed this task.Jun 27 2017, 6:04 PM
hashar triaged this task as Unbreak Now! priority.
Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald Transcript

Mentioned in SAL (#wikimedia-operations) [2017-06-27T18:05:10Z] <hashar> Some CI jobs are broken with "tidy.so: cannot open shared object file: No such file or directory" see T169004

Yeah the hhvm-tidy package is not installed:

ci-jessie-wikimedia-716095:~$ apt-cache policy hhvm-tidy
hhvm-tidy:
  Installed: (none)
  Candidate: 0.1.3~jessie2
  Version table:
     0.1.3~jessie2 0
       1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
     0.1.3~jessie1 0
          1 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/experimental amd64 Packages

Change 361706 had a related patch set uploaded (by Hashar; owner: Hashar):
[operations/puppet@production] Remove libtidy-dev from nodepool instances

https://gerrit.wikimedia.org/r/361706

Change 361706 merged by RobH:
[operations/puppet@production] Remove libtidy-dev from nodepool instances

https://gerrit.wikimedia.org/r/361706

Mentioned in SAL (#wikimedia-releng) [2017-06-27T18:18:41Z] <hashar> Regenerating Jessie nodepool image to hopefulyl bring back hhvm-tidy package - T169004

hhvm-tidy depends on libtidy-0.99-0

libtidy-dev is provided by sury with Breaks: libtidy-0.99-0 and hence it get installed over hhvm-tidy.

libtidy-dev:
  Installed: (none)
  Candidate: 1:5.2.0-1.1+0~20160628154528.2+jessie~1.gbpabdd25
  Version table:
     1:5.2.0-1.1+0~20160628154528.2+jessie~1.gbpabdd25 0
        500 https://packages.sury.org/php/ jessie/main amd64 Packages
     20091223cvs-1.4+deb8u1 0
        500 http://mirrors.wikimedia.org/debian/ jessie/main amd64 Packages
        500 http://security.debian.org/ jessie/updates/main amd64 Packages

We need to fix up the apt pinning for sury.

Mentioned in SAL (#wikimedia-releng) [2017-06-27T18:31:53Z] <hashar> Image snapshot-ci-jessie-1498587497 in wmflabs-eqiad is ready - T169004

hashar closed this task as Resolved.Jun 27 2017, 6:47 PM

Got broken when https://gerrit.wikimedia.org/r/#/c/342635/ installed libtidy-dev which ends up uninstall hhvm-tidy. Fixed by no more installing libtidy-dev. We will need it eventually but that will be the subject of T169008: Figure out the package conflict between libtidy-dev from sury and hhvm-tidy.

I have poked wikitech-l about it:

Hello,

Jenkins jobs relying on HHVM had troubles today exiting with:

  /usr/lib/x86_64-linux-gnu/hhvm/extensions/20150212/tidy.so:
  cannot open shared object file: No such file or directory

The root cause is installing libtidy-dev ends up uninstall the HHVM tidy extension (hhvm-tidy).

The issue occurred between 16:20 UTC and 18:45 UTC.  To the best of my knowledge it is fully solved.

Sorry for the inconvenience :(


Ref:
https://phabricator.wikimedia.org/T169004
Follow up task:
https://phabricator.wikimedia.org/T169008

-- 
Antoine "hashar" Musso