Page MenuHomePhabricator
Create Task
Maniphest T401451

CI build - missing patch & mysqli?
Closed, ResolvedPublic
Actions

Description

We are trying to update our vendor handling in our CRM repository but the new method requires

  1. mysqli - this is *should* now but will become *must* soon - this is being reported as ext-mysqli is not installed
  2. patch - when composer attempts to run patches with this plugin it fails with
patch '-p4' --no-backup-if-mismatch -d '/src/wikimedia/fundraising/crm/vendor/zetacomponents/mail' < '/src/wikimedia/fundraising/crm/core/tools/patches/raw/zetacomponents/mail/100-filename.patch'
sh: 1: patch: not found

I feel I have confirmed the issue is not with the patch file but rather with the patch command being missing or not in the path (probably the latter as @greg checked & says it is in https://docker-registry.wikimedia.org/dev/fundraising-civicrm-bookworm-php82/tags/)

This is the current console output https://integration.wikimedia.org/ci/job/wikimedia-fundraising-civicrm-bookworm/1565/console

Details

Other Assignee
hashar
Related Changes in Gerrit:
Show related patches Customize query in gerrit

Related Objects

Event Timeline

Eileenmcnaughton created this task.Aug 7 2025, 9:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 7 2025, 9:53 PM
greg added a project: Release-Engineering-Team.Aug 7 2025, 10:17 PM

Kindly asking for help diagnosing this Release-Engineering-Team.

dancy changed the task status from Open to In Progress.Aug 7 2025, 10:34 PM
dancy claimed this task.
dancy subscribed.

I'll see what I can do.

gerritbot added a comment.Aug 7 2025, 10:55 PM

Change #1176549 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[integration/config@master] civicrm: Add `patch` package

https://gerrit.wikimedia.org/r/1176549

gerritbot added a project: Patch-For-Review.Aug 7 2025, 10:55 PM
gerritbot added a comment.Aug 7 2025, 10:59 PM

Change #1176549 merged by jenkins-bot:

[integration/config@master] civicrm: Add `patch` package

https://gerrit.wikimedia.org/r/1176549

Eileenmcnaughton added a comment.EditedAug 7 2025, 11:02 PM

@dancy a huge thanks - can you also add php-mysqli here?

https://gerrit.wikimedia.org/r/c/integration/config/+/1176549/1/dockerfiles/civicrm/Dockerfile.template

Eileenmcnaughton added a comment.Aug 7 2025, 11:03 PM

Also - what is the publish process / timeline?

Stashbot added a comment.Aug 7 2025, 11:14 PM

Mentioned in SAL (#wikimedia-releng) [2025-08-07T23:14:02Z] <dancy> Update wikimedia-fundraising-civicrm-bookworm job to use civicrm:0.6 image (T401451)

gerritbot added a comment.Aug 7 2025, 11:14 PM

Change #1176551 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):

[integration/config@master] jjb: Update civicrm to 0.6

https://gerrit.wikimedia.org/r/1176551

dancy added a comment.Aug 7 2025, 11:18 PM
In T401451#11070058, @Eileenmcnaughton wrote:

@dancy a huge thanks - can you also add php-mysqli here?

https://gerrit.wikimedia.org/r/c/integration/config/+/1176549/1/dockerfiles/civicrm/Dockerfile.template

I broke this request into two parts. One for patch since it's straightforward. I (or someone else) will test out the mysqli part later.

In T401451#11070059, @Eileenmcnaughton wrote:

Also - what is the publish process / timeline?

patch is live now. I'm wiped out for the day, and tomorrow is a holiday, so I won't be able to work on the mysqli part in a timely manner. Monday unless someone beats me to it.

Eileenmcnaughton added a comment.Aug 7 2025, 11:19 PM

@dancy thank you very much!

gerritbot added a comment.Aug 11 2025, 9:15 AM

Change #1176551 merged by jenkins-bot:

[integration/config@master] jjb: Update civicrm to 0.6

https://gerrit.wikimedia.org/r/1176551

hashar subscribed.Aug 11 2025, 9:29 AM

The mysqli requirement comes from https://github.com/civicrm/civicrm-core/commit/595f9de96cdfb68f229571551f593c2f5b13669d which was in CiviCRM 6.0.0 when we apparently use 6.3.1. So I am confused as to why it was not a problem previously :]

Note: our image also has php-mysql and php-sqlite3

Maintenance_bot removed a project: Patch-For-Review.Aug 11 2025, 9:30 AM
gerritbot added a comment.Aug 11 2025, 9:35 AM

Change #1177328 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: [civicrm] add php-mysqli

https://gerrit.wikimedia.org/r/1177328

gerritbot added a project: Patch-For-Review.Aug 11 2025, 9:35 AM

Change #1177329 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: update civicrm to 0.7

https://gerrit.wikimedia.org/r/1177329

gerritbot added a comment.Aug 11 2025, 9:44 AM

Change #1177328 merged by jenkins-bot:

[integration/config@master] dockerfiles: [civicrm] add php-mysqli

https://gerrit.wikimedia.org/r/1177328

hashar added a comment.Aug 11 2025, 10:06 AM

I have added php-mysqli to the CI image and update the job to docker-registry.wikimedia.org/releng/civicrm:0.7 (which also includes patch).

I have done a recheck on Gerrit 1175975 CiviCRM 6.6 master (do not merge) but it is currently in merge conflict.

hashar edited projects, added Continuous-Integration-Config; removed Patch-For-Review.Aug 11 2025, 10:07 AM
gerritbot added a comment.Aug 11 2025, 10:09 AM

Change #1177329 merged by jenkins-bot:

[integration/config@master] jjb: update civicrm to 0.7

https://gerrit.wikimedia.org/r/1177329

dancy added a comment.Aug 11 2025, 3:06 PM

Thanks @hashar!

XenoRyet moved this task from Triage to Blocked or not fr-tech on the Fundraising-Backlog board.Aug 11 2025, 8:23 PM
Eileenmcnaughton added a comment.Aug 12 2025, 3:19 AM

Yes - this looks good - the test issue does not seem to relate now

hashar closed this task as Resolved.Aug 12 2025, 6:33 AM

Great, thank you for the confirmation :)

Eileenmcnaughton reopened this task as Open.Aug 14 2025, 6:48 AM

@hashar @dancy -sorry to re-open I just discovered there is a second build being used to run composer.json which does not have mysqli - I see there is an existing issue https://phabricator.wikimedia.org/T226585 - but it's very old & I'm not sure what the recommendation arising from it is

You can see on this patch that the composer job failed although the composer install runs fine on the main build

https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1177559

gerritbot added a comment.Aug 14 2025, 7:17 AM

Change #1178717 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Revert "dockerfiles: [civicrm] add php-mysqli"

https://gerrit.wikimedia.org/r/1178717

gerritbot added a project: Patch-For-Review.Aug 14 2025, 7:17 AM
gerritbot added a comment.Aug 14 2025, 7:21 AM

Change #1178717 merged by jenkins-bot:

[integration/config@master] Revert "dockerfiles: [civicrm] add php-mysqli"

https://gerrit.wikimedia.org/r/1178717

gerritbot added a comment.Aug 14 2025, 7:32 AM

Change #1178721 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: [civicrm] add composer

https://gerrit.wikimedia.org/r/1178721

gerritbot added a comment.Aug 14 2025, 7:34 AM

Change #1178722 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: update civicrm to 0.8

https://gerrit.wikimedia.org/r/1178722

gerritbot added a comment.Aug 14 2025, 7:35 AM

Change #1178721 merged by jenkins-bot:

[integration/config@master] dockerfiles: [civicrm] add composer

https://gerrit.wikimedia.org/r/1178721

gerritbot added a comment.Aug 14 2025, 7:43 AM

Change #1178722 merged by jenkins-bot:

[integration/config@master] jjb: update civicrm to 0.8

https://gerrit.wikimedia.org/r/1178722

gerritbot added a comment.Aug 14 2025, 8:33 AM

Change #1178811 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] dockerfiles: [civicrm] add /run-test from composer image

https://gerrit.wikimedia.org/r/1178811

gerritbot added a comment.Aug 14 2025, 8:50 AM

Change #1178815 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: job to composer test with civicrm image

https://gerrit.wikimedia.org/r/1178815

gerritbot added a comment.Aug 14 2025, 8:50 AM

Change #1178816 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Zuul: [wikimedia/fundraising/crm] composer test with crm env

https://gerrit.wikimedia.org/r/1178816

gerritbot added a comment.Aug 14 2025, 9:17 AM

Change #1178811 merged by jenkins-bot:

[integration/config@master] dockerfiles: [civicrm] add /run-test from composer image

https://gerrit.wikimedia.org/r/1178811

gerritbot added a comment.Aug 14 2025, 9:23 AM

Change #1178815 merged by jenkins-bot:

[integration/config@master] jjb: job to composer test with civicrm image

https://gerrit.wikimedia.org/r/1178815

gerritbot added a comment.Aug 14 2025, 9:23 AM

Change #1178816 merged by jenkins-bot:

[integration/config@master] Zuul: [wikimedia/fundraising/crm] composer test with crm env

https://gerrit.wikimedia.org/r/1178816

hashar added a comment.Aug 14 2025, 9:33 AM

I had to do a bit of refactoring. I have changed the composer test job to use the releng/civicrm image which has the Mysql PHP extension. I have also made it so that Zuul runs composer test first and after it has completed successfully it would run the CiviCRM tests.

gerritbot added a comment.Aug 14 2025, 10:02 AM

Change #1178826 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] jjb: update civicrm to 0.9

https://gerrit.wikimedia.org/r/1178826

gerritbot added a comment.Aug 14 2025, 10:17 AM

Change #1178826 merged by jenkins-bot:

[integration/config@master] jjb: update civicrm to 0.9

https://gerrit.wikimedia.org/r/1178826

hashar added a comment.Aug 14 2025, 10:29 AM

I have removed the composer-php82 job which uses an image that does not have the PHP Mysql extension (nor does it have a MySQL server running).

I have replaced it with a new job wikimedia-fundraising-composer-civicrm which runs the same commands (roughly: composer validate → composer install → composer test) but using the releng/civicrm. That is the same image that is used to run the tests and it does have the mysqli extension.

I have slightly changed the way Zuul runs the tests, instead of running the composer tests and the CiviCRM test suite in parallel, it would now first run the composer test and only when it is successful will it start running the CiviCRM test suite. This means that if parallel-lint / jsonlint fail, Zuul will report a failure faster :)

I ran CI on https://gerrit.wikimedia.org/r/c/wikimedia/fundraising/crm/+/1177559 and it seems to work.

Note: as a result of the above change, the releng/civicrm image now includes composer. When looking at the CiviCRM build I noticed civicrm-buildkit downloads a copy of composer, maybe it can be made to use the version that is already present in the image, but that sounds like an other task :)

10:19:10 Cloning into 'civicrm-buildkit'...
10:19:12 cat: /src/wikimedia/fundraising/crm/civicrm-buildkit/extern/composer.txt: No such file or directory
10:19:12 [[Download composer]]
10:19:12   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
10:19:12                                  Dload  Upload   Total   Spent    Left  Speed
10:19:12 
10:19:12   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
10:19:12   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
10:19:12 
10:19:12 100 2991k  100 2991k    0     0  10.3M      0 --:--:-- --:--:-- --:--:-- 10.3M
10:19:12 Installing dependencies from lock file (including require-dev)
Eileenmcnaughton added a comment.Aug 18 2025, 6:10 AM

It seems to be passing now!

hashar closed this task as Resolved.Aug 18 2025, 9:29 AM
hashar updated Other Assignee, added: hashar.
hashar edited projects, added Essential-Work; removed Patch-For-Review.
In T401451#11085519, @hashar wrote:

...
Note: as a result of the above change, the releng/civicrm image now includes composer. When looking at the CiviCRM build I noticed civicrm-buildkit downloads a copy of composer, maybe it can be made to use the version that is already present in the image, but that sounds like an other task :)

10:19:10 Cloning into 'civicrm-buildkit'...
10:19:12 cat: /src/wikimedia/fundraising/crm/civicrm-buildkit/extern/composer.txt: No such file or directory
10:19:12 [[Download composer]]
10:19:12   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
10:19:12                                  Dload  Upload   Total   Spent    Left  Speed
10:19:12 
10:19:12   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
10:19:12   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
10:19:12 
10:19:12 100 2991k  100 2991k    0     0  10.3M      0 --:--:-- --:--:-- --:--:-- 10.3M
10:19:12 Installing dependencies from lock file (including require-dev)

This comes from bin/civi-download-tools (https://github.com/civicrm/civicrm-buildkit.git) which always install composer. It then capture the version in that extern/composer.txt in order to skip redownloading in the next installation. On CI that is always a fresh start and thus the file never exist :)

It is harmless.

I am marking this as resolved :)

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits