Page MenuHomePhabricator
Create Task
Maniphest T216712

Switch PHP 7.2 packages to an internal component
Closed, ResolvedPublic
Actions

Description

We currently import the php7.2 packages from an external repo and sync them to thirdparty/php72 (complemented with internal builds in component/php72). That worked fine for the initial import, but the latest packages have switched to versions of pcre3 and libzip which are not in stretch (but also part of the external repo). This is problematic, as it complicates updates a lot (we need to be extra careful that not new external dependencies trickle in) and also causes problems on our app servers where we co-host PHP7 and HHVM at the moment. One particular problem I ran into when doing initial tests on a deployment-prep app server is that the new libpcre packages from the external repo use the new style dbgsym mechanism while the existing HHVM debs are using the stock libpcre3-dbg from stretch. This could be fixed with a new HHVM build, but still causes some churn.

In the end using the verbatim debs from the external repo causes more work than also building php7.2 in component/php72, this way we ensure that the Stretch-specific versions of libraries are used and there's less impact on HHVM.

Details

SubjectRepoBranchLines +/-
Create an image for building php packagesintegration/configmaster+22 -0
php72: Switch from thirdparty/php72 to component/php72operations/docker-images/toollabs-imagesmaster+5 -5
Remove sync defition for old PHP 7.2 repooperations/puppetproduction+1 -10
toolforge: switch from thirdparty/php72 to component/php72operations/puppetproduction+1 -1
Switch app servers to component/php72operations/puppetproduction+6 -3
Switch the repo used for php72 packagesmediawiki/vagrantmaster+1 -1
Create component/php72 for stretch-wikimediaoperations/puppetproduction+1 -1
Switch the php72 apt hook to use component/php72operations/puppetproduction+2 -2
Customize query in gerrit

Related Objects
Search...

Event Timeline

MoritzMuehlenhoff created this task.Feb 21 2019, 1:43 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 21 2019, 1:43 PM
MoritzMuehlenhoff added a parent task: T176370: Migrate to PHP 7 in WMF production.Feb 21 2019, 1:44 PM
Paladox subscribed.Feb 21 2019, 1:47 PM
aborrero added projects: Toolforge, cloud-services-team (Kanban).Feb 21 2019, 1:47 PM
Joe mentioned this in T216744: libpcre-related performance opportunities.Feb 22 2019, 7:10 AM
Joe subscribed.Feb 22 2019, 7:39 AM

As far as I can see we're using packages generated by the following source packages:

  1. php - this generates the binary packages php7.2-bcmath php7.2-bz2 php7.2-cli php7.2-common php7.2-curl php7.2-dba php7.2-fpm php7.2-gd php7.2-gmp php7.2-intl php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml
  2. php-redis - this will need to be rebuilt adding php7.2-dev as a build dep *after* the php package is built
  3. php-apcu - ditto
MoritzMuehlenhoff added a comment.Feb 22 2019, 10:18 AM
In T216712#4974896, @Joe wrote:

As far as I can see we're using packages generated by the following source packages:

  1. php - this generates the binary packages php7.2-bcmath php7.2-bz2 php7.2-cli php7.2-common php7.2-curl php7.2-dba php7.2-fpm php7.2-gd php7.2-gmp php7.2-intl php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml

Should be "php7.2"

And in addition also php-defaults

Jdforrester-WMF subscribed.Feb 22 2019, 10:38 AM
Legoktm awarded a token.Feb 22 2019, 9:40 PM
gerritbot subscribed.Feb 26 2019, 10:53 AM

Change 492992 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Switch the php72 apt hook to use component/php72

https://gerrit.wikimedia.org/r/492992

gerritbot added a project: Patch-For-Review.Feb 26 2019, 10:53 AM
gerritbot added a comment.Feb 26 2019, 11:10 AM

Change 492992 merged by Muehlenhoff:
[operations/puppet@production] Switch the php72 apt hook to use component/php72

https://gerrit.wikimedia.org/r/492992

gerritbot added a comment.Feb 26 2019, 11:48 AM

Change 493002 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Create component/php72 for stretch-wikimedia

https://gerrit.wikimedia.org/r/493002

gerritbot added a comment.Feb 26 2019, 11:48 AM

Change 493002 merged by Muehlenhoff:
[operations/puppet@production] Create component/php72 for stretch-wikimedia

https://gerrit.wikimedia.org/r/493002

Stashbot subscribed.Feb 26 2019, 12:14 PM

Mentioned in SAL (#wikimedia-operations) [2019-02-26T12:14:40Z] <moritzm> uploaded php7.2 7.2.15-1+0~20190209065123.16+stretch~1.gbp3ad8c0+wmf1 to component/php72 (T216712)

MoritzMuehlenhoff added a comment.Feb 28 2019, 2:39 PM

The core php72 packages had the extensions rebuilt for 7.2 were all rebuilt (in the correct ordering) within the component/php72. For any further updates to a new 7.2.x release we only need to rebuild src:php7.2 as the ABI of the extension package remains the same. I'll pick some application server in codfw for some tests.

gerritbot added a comment.Feb 28 2019, 4:50 PM

Change 493445 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[mediawiki/vagrant@master] Switch the repo used for php72 packages

https://gerrit.wikimedia.org/r/493445

gerritbot added a comment.Feb 28 2019, 4:55 PM

Change 493451 had a related patch set uploaded (by BryanDavis; owner: Bryan Davis):
[operations/puppet@production] toolforge: switch from thirdparty/php72 to component/php72

https://gerrit.wikimedia.org/r/493451

gerritbot added a comment.Feb 28 2019, 8:39 PM

Change 493445 merged by jenkins-bot:
[mediawiki/vagrant@master] Switch the repo used for php72 packages

https://gerrit.wikimedia.org/r/493445

gerritbot added a comment.Mar 4 2019, 12:57 PM

Change 494212 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Switch app servers to component/php72

https://gerrit.wikimedia.org/r/494212

jbond triaged this task as Medium priority.Mar 4 2019, 3:31 PM
Dzahn subscribed.Mar 5 2019, 5:22 AM
gerritbot added a comment.Mar 5 2019, 1:07 PM

Change 494212 merged by Muehlenhoff:
[operations/puppet@production] Switch app servers to component/php72

https://gerrit.wikimedia.org/r/494212

Stashbot added a comment.Mar 7 2019, 9:21 AM

Mentioned in SAL (#wikimedia-operations) [2019-03-07T09:21:19Z] <moritzm> upgrading mwdebug servers in codfw to component/php72 (T216712)

Stashbot added a comment.Mar 7 2019, 9:47 AM

Mentioned in SAL (#wikimedia-operations) [2019-03-07T09:47:55Z] <moritzm> upgrading mwdebug servers in eqiad to component/php72 (T216712)

Stashbot added a comment.Mar 7 2019, 10:13 AM

Mentioned in SAL (#wikimedia-operations) [2019-03-07T10:13:26Z] <moritzm> upgrading mediawiki canaries to component/php72 (T216712)

MoritzMuehlenhoff added subscribers: bd808, Legoktm.Mar 8 2019, 6:50 AM

@Legoktm , @bd808 : The PHP 7.2 packages from the component/php72 are working fine in the Mediawiki PHP production setup, but there's also two packages that are not being used in production, but which were imported to thirdparty/php72: php-xdebug and php-tideways.

I tried building xdebug, but the configure script bails out with

checking Check for supported PHP versions... configure: error: not supported. Need a PHP version >= 5.5.0 and < 7.2.0 (found 7.2.15-1+0~20190209065123.16+stretch~1.gbp3ad8c0+wmf1)

Is php-xdebug (and php-tideways) used in the stretch-based Toolforge PHP setup? If so, we could build xdebug 2.7.0, which was uploaded to Debian last night and should be compatible with PHP 7.2: https://packages.qa.debian.org/x/xdebug/news/20190307T203909Z.html

Stashbot added a comment.Mar 8 2019, 11:07 AM

Mentioned in SAL (#wikimedia-operations) [2019-03-08T11:07:55Z] <moritzm> uploaded tideways 4.0.7-1+wmf1 for component/php72 (T216712)

Dzahn added a comment.Mar 8 2019, 11:10 AM

phab1002 done in https://gerrit.wikimedia.org/r/c/operations/puppet/+/494885

doc1001 done in https://gerrit.wikimedia.org/r/c/operations/puppet/+/494884 , https://gerrit.wikimedia.org/r/c/operations/puppet/+/494903

bd808 added a comment.Mar 8 2019, 4:40 PM
In T216712#5010593, @MoritzMuehlenhoff wrote:

Is php-xdebug (and php-tideways) used in the stretch-based Toolforge PHP setup? If so, we could build xdebug 2.7.0, which was uploaded to Debian last night and should be compatible with PHP 7.2: https://packages.qa.debian.org/x/xdebug/news/20190307T203909Z.html

php-xdebug is deployed in both MediaWiki-Vagrant and Toolforge today. Both of these environments have local apt repos that we could add packages to if needed, but it would be nice if they could draw from the central repo instead.

$ ssh login.tools.wmflabs.org
$ apt-cache policy php-xdebug
php-xdebug:
  Installed: 2.7.0~rc1+2.6.1+2.5.5-1+0~20190203094445.6+stretch~1.gbpf2664f
  Candidate: 2.7.0~rc1+2.6.1+2.5.5-1+0~20190203094445.6+stretch~1.gbpf2664f
  Version table:
 *** 2.7.0~rc1+2.6.1+2.5.5-1+0~20190203094445.6+stretch~1.gbpf2664f 1001
       1001 http://apt.wikimedia.org/wikimedia stretch-wikimedia/thirdparty/php72 amd64 Packages
        100 /var/lib/dpkg/status
     2.5.0-1 500
        500 http://deb.debian.org/debian stretch/main amd64 Packages
gerritbot added a comment.Mar 8 2019, 5:29 PM

Change 493451 merged by Bstorm:
[operations/puppet@production] toolforge: switch from thirdparty/php72 to component/php72

https://gerrit.wikimedia.org/r/493451

gerritbot added a comment.Mar 8 2019, 7:46 PM

Change 495291 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[operations/docker-images/toollabs-images@master] php72: Switch from thirdparty/php72 to component/php72

https://gerrit.wikimedia.org/r/495291

Legoktm mentioned this in rODIT4cd908375e14: php72: Switch from thirdparty/php72 to component/php72.Mar 8 2019, 8:23 PM
Stashbot added a comment.Mar 11 2019, 2:09 PM

Mentioned in SAL (#wikimedia-operations) [2019-03-11T14:09:04Z] <moritzm> importing build of PHP 7.2.16 for component/php72 (T216712)

Stashbot added a comment.Mar 12 2019, 8:48 AM

Mentioned in SAL (#wikimedia-operations) [2019-03-12T08:48:09Z] <moritzm> upgrading app servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712)

Stashbot added a comment.Mar 12 2019, 10:17 AM

Mentioned in SAL (#wikimedia-operations) [2019-03-12T10:17:40Z] <moritzm> upgrading API servers/job runners servers in codfw to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712)

Stashbot added a comment.Mar 12 2019, 11:11 AM

Mentioned in SAL (#wikimedia-operations) [2019-03-12T11:11:30Z] <moritzm> upgrading API servers/job runners servers in eqiad to component/php72 / PHP 7.2.16 (combined with glibc/OpenSSL updates) (T216712)

MoritzMuehlenhoff added a comment.Mar 13 2019, 9:06 AM

Production has been switched to the new component, all working fine. The new approach is also fairly straightforward, the upgrade from 7.2.15 to 7.2.16 was a straightforward import/build of the new php7.2 source package with all extensions continuing to work fine.

The only missing bit is to remove thirdparty/php72 from apt.wikimedia.org to prevent people accidentally using an unmaintained/insecure repo section. I need to doublecheck whether all PHP nodes in the Toolforge stretch grid are migrated to component/php72.

Paladox added a comment.Mar 13 2019, 9:21 AM

Phabricator uses thirdparty/php72 I think.

MoritzMuehlenhoff added a comment.Mar 13 2019, 9:31 AM

Actually, I had only been looking at servers with php7.2-fpm installed, the deployment, maintenance and snapshot hosts will also need to be converted to the component.

MoritzMuehlenhoff added a comment.Mar 13 2019, 9:32 AM
In T216712#5020202, @Paladox wrote:

Phabricator uses thirdparty/php72 I think.

This was already converted last week.

MoritzMuehlenhoff added a subtask: T218193: Switch dumps to component/php7.2.Mar 13 2019, 10:24 AM
MoritzMuehlenhoff claimed this task.Mar 13 2019, 11:32 AM
bd808 added a comment.Mar 14 2019, 5:00 AM

I switched the Toolforge PHP 7.2 image: https://gerrit.wikimedia.org/r/#/c/operations/docker-images/toollabs-images/+/496102/

Joe mentioned this in T217938: Beta Cluster does not have php7.0-redis available.Mar 18 2019, 12:27 PM
gerritbot added a comment.Mar 22 2019, 1:53 PM

Change 498376 had a related patch set uploaded (by Muehlenhoff; owner: Muehlenhoff):
[operations/puppet@production] Remove sync defition for old PHP 7.2 repo

https://gerrit.wikimedia.org/r/498376

gerritbot added a comment.Mar 22 2019, 1:56 PM

Change 498376 merged by Muehlenhoff:
[operations/puppet@production] Remove sync defition for old PHP 7.2 repo

https://gerrit.wikimedia.org/r/498376

gerritbot added a comment.Apr 1 2019, 6:53 AM

Change 500381 had a related patch set uploaded (by Giuseppe Lavagetto; owner: Giuseppe Lavagetto):
[integration/config@master] Create an image for building php packages

https://gerrit.wikimedia.org/r/500381

MoritzMuehlenhoff closed subtask T218193: Switch dumps to component/php7.2 as Resolved.Apr 1 2019, 7:29 AM
bd808 mentioned this in T212045: Vagrant has Xdebug no longer enabled, preventing debugging and code coverage generation.Apr 25 2019, 3:38 AM
bd808 mentioned this in T220406: Package XDebug 2.9 for apt.wikimedia.org.Apr 26 2019, 2:28 AM
Mainframe98 mentioned this in T221923: Please import php-xdebug to apt.wm.o component/php72.Apr 26 2019, 7:59 AM
Joe closed this task as Resolved.May 10 2019, 3:36 PM
gerritbot added a comment.Dec 14 2019, 10:13 AM

Change 495291 abandoned by Legoktm:
php72: Switch from thirdparty/php72 to component/php72

https://gerrit.wikimedia.org/r/495291

gerritbot added a comment.Jun 17 2020, 2:08 PM

Change 500381 abandoned by Hashar:
Create an image for building php packages

Reason:
Per my previous comment. Else I guess a better fit would be under operations/docker-images/production-images

https://gerrit.wikimedia.org/r/500381

Maintenance_bot removed a project: Patch-For-Review.Jun 17 2020, 2:11 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL