Page MenuHomePhabricator

Switch PHP 7.2 packages to an internal component
Closed, ResolvedPublic

Description

We currently import the php7.2 packages from an external repo and sync them to thirdparty/php72 (complemented with internal builds in component/php72). That worked fine for the initial import, but the latest packages have switched to versions of pcre3 and libzip which are not in stretch (but also part of the external repo). This is problematic, as it complicates updates a lot (we need to be extra careful that not new external dependencies trickle in) and also causes problems on our app servers where we co-host PHP7 and HHVM at the moment. One particular problem I ran into when doing initial tests on a deployment-prep app server is that the new libpcre packages from the external repo use the new style dbgsym mechanism while the existing HHVM debs are using the stock libpcre3-dbg from stretch. This could be fixed with a new HHVM build, but still causes some churn.

In the end using the verbatim debs from the external repo causes more work than also building php7.2 in component/php72, this way we ensure that the Stretch-specific versions of libraries are used and there's less impact on HHVM.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 21 2019, 1:43 PM
Joe added a subscriber: Joe.Feb 22 2019, 7:39 AM

As far as I can see we're using packages generated by the following source packages:

  1. php - this generates the binary packages php7.2-bcmath php7.2-bz2 php7.2-cli php7.2-common php7.2-curl php7.2-dba php7.2-fpm php7.2-gd php7.2-gmp php7.2-intl php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml
  2. php-redis - this will need to be rebuilt adding php7.2-dev as a build dep *after* the php package is built
  3. php-apcu - ditto

As far as I can see we're using packages generated by the following source packages:

  1. php - this generates the binary packages php7.2-bcmath php7.2-bz2 php7.2-cli php7.2-common php7.2-curl php7.2-dba php7.2-fpm php7.2-gd php7.2-gmp php7.2-intl php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml

Should be "php7.2"

And in addition also php-defaults

Change 492992 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Switch the php72 apt hook to use component/php72

https://gerrit.wikimedia.org/r/492992

Change 492992 merged by Muehlenhoff:
[operations/puppet@production] Switch the php72 apt hook to use component/php72

https://gerrit.wikimedia.org/r/492992

Change 493002 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Create component/php72 for stretch-wikimedia

https://gerrit.wikimedia.org/r/493002

Change 493002 merged by Muehlenhoff:
[operations/puppet@production] Create component/php72 for stretch-wikimedia

https://gerrit.wikimedia.org/r/493002

Mentioned in SAL (#wikimedia-operations) [2019-02-26T12:14:40Z] <moritzm> uploaded php7.2 7.2.15-1+0~20190209065123.16+stretch~1.gbp3ad8c0+wmf1 to component/php72 (T216712)

The core php72 packages had the extensions rebuilt for 7.2 were all rebuilt (in the correct ordering) within the component/php72. For any further updates to a new 7.2.x release we only need to rebuild src:php7.2 as the ABI of the extension package remains the same. I'll pick some application server in codfw for some tests.

Change 493445 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@master] Switch the repo used for php72 packages

https://gerrit.wikimedia.org/r/493445

Change 493451 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] toolforge: switch from thirdparty/php72 to component/php72

https://gerrit.wikimedia.org/r/493451

Change 493445 merged by jenkins-bot:
[mediawiki/vagrant@master] Switch the repo used for php72 packages

https://gerrit.wikimedia.org/r/493445

Change 494212 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Switch app servers to component/php72

https://gerrit.wikimedia.org/r/494212

jbond triaged this task as Normal priority.Mar 4 2019, 3:31 PM

Change 494212 merged by Muehlenhoff:
[operations/puppet@production] Switch app servers to component/php72

https://gerrit.wikimedia.org/r/494212

Mentioned in SAL (#wikimedia-operations) [2019-03-07T09:21:19Z] <moritzm> upgrading mwdebug servers in codfw to component/php72 (T216712)

Mentioned in SAL (#wikimedia-operations) [2019-03-07T09:47:55Z] <moritzm> upgrading mwdebug servers in eqiad to component/php72 (T216712)

Mentioned in SAL (#wikimedia-operations) [2019-03-07T10:13:26Z] <moritzm> upgrading mediawiki canaries to component/php72 (T216712)

@Legoktm , @bd808 : The PHP 7.2 packages from the component/php72 are working fine in the Mediawiki PHP production setup, but there's also two packages that are not being used in production, but which were imported to thirdparty/php72: php-xdebug and php-tideways.

I tried building xdebug, but the configure script bails out with

checking Check for supported PHP versions... configure: error: not supported. Need a PHP version >= 5.5.0 and < 7.2.0 (found 7.2.15-1+0~20190209065123.16+stretch~1.gbp3ad8c0+wmf1)

Is php-xdebug (and php-tideways) used in the stretch-based Toolforge PHP setup? If so, we could build xdebug 2.7.0, which was uploaded to Debian last night and should be compatible with PHP 7.2: https://packages.qa.debian.org/x/xdebug/news/20190307T203909Z.html

Mentioned in SAL (#wikimedia-operations) [2019-03-08T11:07:55Z] <moritzm> uploaded tideways 4.0.7-1+wmf1 for component/php72 (T216712)

bd808 added a comment.Mar 8 2019, 4:40 PM

Is php-xdebug (and php-tideways) used in the stretch-based Toolforge PHP setup? If so, we could build xdebug 2.7.0, which was uploaded to Debian last night and should be compatible with PHP 7.2: https://packages.qa.debian.org/x/xdebug/news/20190307T203909Z.html

php-xdebug is deployed in both MediaWiki-Vagrant and Toolforge today. Both of these environments have local apt repos that we could add packages to if needed, but it would be nice if they could draw from the central repo instead.

$ ssh login.tools.wmflabs.org
$ apt-cache policy php-xdebug
php-xdebug:
  Installed: 2.7.0~rc1+2.6.1+2.5.5-1+0~20190203094445.6+stretch~1.gbpf2664f
  Candidate: 2.7.0~rc1+2.6.1+2.5.5-1+0~20190203094445.6+stretch~1.gbpf2664f
  Version table:
 *** 2.7.0~rc1+2.6.1+2.5.5-1+0~20190203094445.6+stretch~1.gbpf2664f 1001
       1001 http://apt.wikimedia.org/wikimedia stretch-wikimedia/thirdparty/php72 amd64 Packages
        100 /var/lib/dpkg/status
     2.5.0-1 500
        500 http://deb.debian.org/debian stretch/main amd64 Packages

Change 493451 merged by Bstorm:
[operations/puppet@production] toolforge: switch from thirdparty/php72 to component/php72

https://gerrit.wikimedia.org/r/493451

Change 495291 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[operations/docker-images/toollabs-images@master] php72: Switch from thirdparty/php72 to component/php72

https://gerrit.wikimedia.org/r/495291

Mentioned in SAL (#wikimedia-operations) [2019-03-11T14:09:04Z] <moritzm> importing build of PHP 7.2.16 for component/php72 (T216712)

Mentioned in SAL (#wikimedia-operations) [2019-03-12T08:48:09Z] <moritzm> upgrading app servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712)

Mentioned in SAL (#wikimedia-operations) [2019-03-12T10:17:40Z] <moritzm> upgrading API servers/job runners servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712)

Mentioned in SAL (#wikimedia-operations) [2019-03-12T11:11:30Z] <moritzm> upgrading API servers/job runners servers in eqiad to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712)

Production has been switched to the new component, all working fine. The new approach is also fairly straightforward, the upgrade from 7.2.15 to 7.2.16 was a straightforward import/build of the new php7.2 source package with all extensions continuing to work fine.

The only missing bit is to remove thirdparty/php72 from apt.wikimedia.org to prevent people accidentally using an unmaintained/insecure repo section. I need to doublecheck whether all PHP nodes in the Toolforge stretch grid are migrated to component/php72.

Phabricator uses thirdparty/php72 I think.

Actually, I had only been looking at servers with php7.2-fpm installed, the deployment, maintenance and snapshot hosts will also need to be converted to the component.

Phabricator uses thirdparty/php72 I think.

This was already converted last week.

Change 498376 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove sync defition for old PHP 7.2 repo

https://gerrit.wikimedia.org/r/498376

Change 498376 merged by Muehlenhoff:
[operations/puppet@production] Remove sync defition for old PHP 7.2 repo

https://gerrit.wikimedia.org/r/498376

Change 500381 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[integration/config@master] Create an image for building php packages

https://gerrit.wikimedia.org/r/500381

Joe closed this task as Resolved.May 10 2019, 3:36 PM