Page MenuHomePhabricator
Create Task
Maniphest T250515

Please provide our special component/php72 in buster-wikimedia
Closed, ResolvedPublic
Actions

Description

This will allow us to run like-for-like versions of PHP in CI on buster.

Details

SubjectRepoBranchLines +/-
jjb: update jobs for quibble-buster-php72:0.0.45-s4integration/configmaster+1 -1
dockerfiles: update quibble-buster-php72 packagesintegration/configmaster+6 -0
dockerfiles: [quibble-buster-php72] Switch to Wikimedia's version of PHP 7.2integration/configmaster+7 -16
Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedNoneT248925 Make MediaWiki release tarball compatible with PHP 8.0
 ResolvedJdforrester-WMFT300463 Make PHP 8.0 voting on MW master
 ResolvedJdforrester-WMFT313563 Bump lcobucci/jwt & league/uri-components for php8
 ResolvedJdforrester-WMFT313564 Bump onoi/message-reporter in vendor.git to 1.4.2 for php 8 support
 ResolvedJdforrester-WMFT247658 Make Wikimedia Production MediaWiki compatible with PHP 7.4
 Resolved• toanT243590 Fix WikibaseDataModel CI for php 7.4
 ResolvedLucas_Werkmeister_WMDET316923 Restore skipped test in ReferenceListTest.php
 ResolvedJoeT318918 Undeploy patch to use old PHP serialization in PHP 7.4
 ResolvedJdforrester-WMFT264168 Drop PHP 7.2 support from Wikibase master branch, once Wikimedia production is on 7.4
 ResolvedLadsgroupT270740 Drop hacky support for doctrine/dbal class renaming
 InvalidNoneT303505 [S] Remove Deprecated methods "serialize" and "unserialize" after php production upgrade to PHP 7.4
 ResolvedReedyT251043 Cleanup css-sanitizer when it only requires PHP >= 7.4
 OpenNoneT166010 The Great Namespaceization Effort
 ResolvedtstarlingT277618 var_dump() on various objects writes gigabytes of data and takes minutes to run
 ResolvedJdforrester-WMFT261872 Drop PHP 7.2 & 7.3 support from MediaWiki master branch, once Wikimedia production is on 7.4
 StalledNoneT302086 Set scap minimum python version to 3.7
 ResolvedNoneT247045 Migrate all of production metal and VMs to Buster or later
 ResolvedPRODUCTION ERRORLegoktmT293568 PHP Notice: Undefined offset in wikimedia/remex-html when rendering rest.php error page
 ResolvedtstarlingT297667 mysqli/mysqlnd memory leak
 ResolvedJoeT271736 Migrate WMF production from PHP 7.2 to PHP 7.4
 ResolvedKrinkleT248191 Can't reopen table in wikidb-unittest_ (from SpecialPageFatalTest)
 ResolvedhasharT278203 Migrate all CI jobs from stretch to buster or later and drop stretch testing support
 ResolvedJdforrester-WMFT252432 Drop MediaWiki testing in stretch and instead test only in buster
 ResolvedDzahnT245757 Upgrade MediaWiki clusters to Debian Buster (debian 10)
 ResolvedMoritzMuehlenhoffT250515 Please provide our special component/php72 in buster-wikimedia

Event Timeline

Jdforrester-WMF created this task.Apr 17 2020, 6:36 PM
Jdforrester-WMF triaged this task as Low priority.Apr 17 2020, 6:39 PM
Jdforrester-WMF mentioned this in T250513: Split out all selenium test runs into a buster-based job that runs newer Chromium.Apr 17 2020, 9:35 PM
Jdforrester-WMF added a project: Packaging.May 22 2020, 4:47 PM
MoritzMuehlenhoff subscribed.May 29 2020, 3:27 PM

That's a total of 20 packages needing a rebuild, realistically we'll only approach this once we move production to Buster.

Jdforrester-WMF added a comment.May 29 2020, 7:41 PM
In T250515#6177325, @MoritzMuehlenhoff wrote:

That's a total of 20 packages needing a rebuild, realistically we'll only approach this once we move production to Buster.

I'm happy to do the work for you if you point to the git repo where this is managed. It's blocking a pretty big pain point for CI (because Chromium is no longer updated in stretch).

Legoktm subscribed.May 31 2020, 7:16 AM

Can we use sury's PHP apt repo in the meantime?

MoritzMuehlenhoff added a comment.Jun 3 2020, 7:39 AM

What @Legoktm said, to run CI tests on Buster and PHP 7.2 best to use the sury.org repository. Our component for stretch-wikimedia is a rebuild of sury.org which fixes a build issue with our mariadb version on stretch and which ensures that only stock libraries from Debian stretch are used during the build (the sury.org repo covers more distros and also backports some base libs). In the absence of production actually running PHP 7.2 on buster, using sury.org is the best approximation.

Jdforrester-WMF added a comment.Jun 3 2020, 3:46 PM

OK, that works.

Jdforrester-WMF edited parent tasks, added: T245757: Upgrade MediaWiki clusters to Debian Buster (debian 10); removed: T250514: Create buster-based images for quibble.Jun 3 2020, 3:55 PM
hashar mentioned this in T257879: Drop PHP 7.2 support in MediaWiki 1.35; require PHP 7.3.19.Jul 30 2020, 3:13 PM
jijiki moved this task from Incoming 🐫 to 🔦Unused2 on the serviceops board.Aug 17 2020, 11:45 PM
Dzahn mentioned this in T264991: Upgrade the MediaWiki servers to ICU 63.Oct 9 2020, 11:25 PM
Dzahn subscribed.

T264991#6532447

MoritzMuehlenhoff claimed this task.Oct 12 2020, 7:21 AM
In T250515#6177325, @MoritzMuehlenhoff wrote:

realistically we'll only approach this once we move production to Buster.

This is happening soon, so once available we can also switch CI to it (if it's currently using the 7.2 packages from sury.org, not sure)

gerritbot added a comment.Oct 13 2020, 9:59 PM

Change 633837 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] [DNM] dockerfiles: [quibble-buster-php72] Switch to Wikimedia's version of PHP 7.2

https://gerrit.wikimedia.org/r/633837

gerritbot added a project: Patch-For-Review.Oct 13 2020, 9:59 PM
gerritbot added a comment.Dec 5 2020, 1:49 AM

Change 633837 merged by jenkins-bot:
[integration/config@master] dockerfiles: [quibble-buster-php72] Switch to Wikimedia's version of PHP 7.2

https://gerrit.wikimedia.org/r/633837

Jdforrester-WMF closed this task as Resolved.Dec 5 2020, 1:50 AM

Thanks!

Maintenance_bot removed a project: Patch-For-Review.Dec 5 2020, 2:10 AM
hashar reopened this task as Open.Dec 8 2020, 3:18 PM
hashar subscribed.

Summary: buster-wikimedia components/php72 provides php-wikidiff2 1.8.1 instead of 1.10.0

I have compared the Strech versus Buster images using:

docker run --rm -it --entrypoint=apt docker-registry.discovery.wmnet/releng/quibble-stretch-php72 list 'php*'
docker run --rm -it --entrypoint=apt docker-registry.discovery.wmnet/releng/quibble-buster-php72 list 'php*'

php-wikidiff2 on Stretch is 1.10.0-1~wmf1 while Buster provides the outdated 1.8.1-1~wmf1+buster1. We require a more recent one for some tests, notably for the api-testing suite.

All the other packages match :]

Looking inside the containers:

Stretch
php-wikidiff2:
  Installed: 1.10.0-1~wmf1
  Candidate: 1.10.0-1~wmf1
  Version table:
 *** 1.10.0-1~wmf1 1001
       1001 http://apt.wikimedia.org/wikimedia stretch-wikimedia/main amd64 Packages
        100 /var/lib/dpkg/status
     1.8.1-1~wmf1 1001
       1001 http://apt.wikimedia.org/wikimedia stretch-wikimedia/component/php72 amd64 Packages
     1.4.1-1 500
        500 http://mirrors.wikimedia.org/debian stretch/main amd64 Packages
Buster
# apt-cache policy php-wikidiff2
php-wikidiff2:
  Installed: 1.8.1-1~wmf1+buster1
  Candidate: 1.8.1-1~wmf1+buster1
  Version table:
     1.10.0-1~bpo10+1 100
        100 http://mirrors.wikimedia.org/debian buster-backports/main amd64 Packages
 *** 1.8.1-1~wmf1+buster1 1001
       1001 http://apt.wikimedia.org/wikimedia buster-wikimedia/component/php72 amd64 Packages
        100 /var/lib/dpkg/status
     1.7.3-2 500
        500 http://mirrors.wikimedia.org/debian buster/main amd64 Packages

I guess the issue is that when php-wikidiff2 1.10.0 got build for Stretch it got uploaded to main instead of component/php72. When later building for Buster, the same versions used in component/php72 got used and we end up with the outdated version.

There is the version from buster-backports, though it would not necessarily match production and we found out that backports magically disappear before the EOL of the distribution. So probably buster-backports should be removed from our CI images but that is a different problem.

MoritzMuehlenhoff added a comment.Dec 8 2020, 3:51 PM
In T250515#6676402, @hashar wrote:

Summary: buster-wikimedia components/php72 provides php-wikidiff2 1.8.1 instead of 1.10.0

I have compared the Strech versus Buster images using:

docker run --rm -it --entrypoint=apt docker-registry.discovery.wmnet/releng/quibble-stretch-php72 list 'php*'
docker run --rm -it --entrypoint=apt docker-registry.discovery.wmnet/releng/quibble-buster-php72 list 'php*'

php-wikidiff2 on Stretch is 1.10.0-1~wmf1 while Buster provides the outdated 1.8.1-1~wmf1+buster1. We require a more recent one for some tests, notably for the api-testing suite.

That's actually a bug, will update the package tomorrow.

Jdforrester-WMF added a comment.Dec 8 2020, 4:57 PM
In T250515#6676486, @MoritzMuehlenhoff wrote:
In T250515#6676402, @hashar wrote:

Summary: buster-wikimedia components/php72 provides php-wikidiff2 1.8.1 instead of 1.10.0

I have compared the Strech versus Buster images using:

docker run --rm -it --entrypoint=apt docker-registry.discovery.wmnet/releng/quibble-stretch-php72 list 'php*'
docker run --rm -it --entrypoint=apt docker-registry.discovery.wmnet/releng/quibble-buster-php72 list 'php*'

php-wikidiff2 on Stretch is 1.10.0-1~wmf1 while Buster provides the outdated 1.8.1-1~wmf1+buster1. We require a more recent one for some tests, notably for the api-testing suite.

That's actually a bug, will update the package tomorrow.

Also a PHP 8.0 release of wikidiff2 is coming out today, so both packages will want updating.

Stashbot added a comment.Dec 9 2020, 4:05 PM

Mentioned in SAL (#wikimedia-operations) [2020-12-09T16:05:00Z] <moritzm> importing wikidiff2 1.10.0-1~wmf1+buster1 to component/php72 T250515

MoritzMuehlenhoff added a comment.Dec 9 2020, 4:13 PM
In T250515#6676486, @MoritzMuehlenhoff wrote:

That's actually a bug, will update the package tomorrow.

Fixed.

gerritbot added a comment.Feb 8 2021, 11:59 AM

Change 662663 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] dockerfiles: update quibble-buster-php72 packages

https://gerrit.wikimedia.org/r/662663

gerritbot added a project: Patch-For-Review.Feb 8 2021, 11:59 AM

Change 662664 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] jjb: update jobs for quibble-buster-php72:0.0.45-s4

https://gerrit.wikimedia.org/r/662664

gerritbot added a comment.Feb 8 2021, 12:01 PM

Change 662663 merged by jenkins-bot:
[integration/config@master] dockerfiles: update quibble-buster-php72 packages

https://gerrit.wikimedia.org/r/662663

gerritbot added a comment.Mar 16 2021, 5:14 PM

Change 662664 abandoned by Jforrester:
[integration/config@master] jjb: update jobs for quibble-buster-php72:0.0.45-s4

Reason:
Over-taken by 2b684def8.

https://gerrit.wikimedia.org/r/662664

Maintenance_bot removed a project: Patch-For-Review.Mar 16 2021, 6:12 PM
hashar closed this task as Resolved.Mar 29 2021, 12:38 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL