Yes, you can install other packages using https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Installing_Apt_packages

Note though that the binaries will not be installed in the usual path, so make sure to use $PATH for the resolution (like https://docs.rs/command-run/latest/command_run/struct.Command.html#structfield.program uses).

I added an Aptfile as described and rebuild the image, but:

tools.mix-n-match@tools-sgebastion-10:~/mixnmatch_rs$ toolforge jobs run --mem 2000Mi --cpu 1 --wait --mount=all --image tool-mix-n-match/tool-mix-n-match:latest --command "php --version" single
ERROR: job 'single' failed:
+-------------+------------------------------------------------------------------+
| Job name:   | single                                                           |
+-------------+------------------------------------------------------------------+
| Command:    | launcher php --version                                           |
+-------------+------------------------------------------------------------------+
| Job type:   | one-off                                                          |
+-------------+------------------------------------------------------------------+
| Image:      | tool-mix-n-match/tool-mix-n-match:latest                         |
+-------------+------------------------------------------------------------------+
| File log:   | no                                                               |
+-------------+------------------------------------------------------------------+
| Output log: |                                                                  |
+-------------+------------------------------------------------------------------+
| Error log:  |                                                                  |
+-------------+------------------------------------------------------------------+
| Emails:     | none                                                             |
+-------------+------------------------------------------------------------------+
| Resources:  | mem: 2000Mi, cpu: 1                                              |
+-------------+------------------------------------------------------------------+
| Mounts:     | all                                                              |
+-------------+------------------------------------------------------------------+
| Retry:      | no                                                               |
+-------------+------------------------------------------------------------------+
| Status:     | Failed                                                           |
+-------------+------------------------------------------------------------------+
| Hints:      | Last run at 2024-03-12T14:23:38Z. Pod in 'Failed' phase. State   |
|             | 'terminated'. Reason 'Error'. Started at '2024-03-12T14:23:39Z'. |
|             | Finished at '2024-03-12T14:23:39Z'. Exit code '127'.             |
+-------------+------------------------------------------------------------------+

I also tried commands like "sh -c 'php --version'" but no success.

I saw the manual for the interactive shell but that's only for the webservice?

Ah, I found /layers/fagiani_apt/apt/usr/bin/php8.1, I guess it's just not symlinked anywhere?

In T359937#9623701, @Magnus wrote:

Ah, I found /layers/fagiani_apt/apt/usr/bin/php8.1, I guess it's just not symlinked anywhere?

Might be yes :/, we don't run the deb package scripts (as we are just extracting them, it's not possible to install them as usual), so some things might be missing, the binary should be in the path though (I recommend avoiding to use the hardcodded path, as it depends on the buildpack, the only long-term assurance is that it will be in $PATH).

Would it be possible to add a symlink in $PATH somewhere? php is stable, php8.1 maybe less so

In T359937#9623782, @Magnus wrote:

Would it be possible to add a symlink in $PATH somewhere? php is stable, php8.1 maybe less so

The issue there is that we would not know which binary to simlink as php, so if the php version gets upgraded, or you install more than one version (not sure if that's possible with php though), then there's no way to differentiate it.

I'm open to ideas though.

Looks like the php setup is incomplete. Running php8.1 path/to/my/php/script dies with Fatal error: Uncaught Error: Class "mysqli" not found.

It looks like, for this tool, the buildservice is not functional. However, I can't seem to build it another way: on tools-sgebastion it dies for insufficient resources, gridengine is shut down, and building in a toolforge job didn't work anymore (can't find the phabricator ticket now, but basically it said to use buildservice).

Do you have a solution for me? I could try to cross-compile on my Mac for toolforge linux but that seems a bit out there

In T359937#9623974, @Magnus wrote:

Looks like the php setup is incomplete. Running php8.1 path/to/my/php/script dies with Fatal error: Uncaught Error: Class "mysqli" not found.

The PHP mysql extension needs to be installed separately from the php-mysql package.

Thanks @taavi Is there a list of packages required for a toolforge-equivalent php setup?

@taavi I added php-mysql to the Aptfile and rebuild the image, same error.

I undestand and applaud the drive to use Docker and k8s and buildservices etc but now I, as a volunteer, have to invest considerable time and effort piecing together how to keep my code running as it was running before y'all changed things. This is a bit frustrating.

In T359937#9623995, @Magnus wrote:

@taavi I added php-mysql to the Aptfile and rebuild the image, same error.

I undestand and applaud the drive to use Docker and k8s and buildservices etc but now I, as a volunteer, have to invest considerable time and effort piecing together how to keep my code running as it was running before y'all changed things. This is a bit frustrating.

It's understandable yes, there's some changes needed. The great advantage is that the previous setup was unsustainable, and focused on usage patterns that did not make it easier to sustain either, with the new one, unless you are doing something very special, will make things easier to maintain for longer.

Anyhow, I think that the issue with the php module might be that the path that php loads the modules from seems to be hardcodded in the php binary itself, and the modules are installed under /layers/fagiani_apt/apt/usr/lib/php/20210902, not /usr/lib/php/20210902, so you might need to create a php.ini to change the php behavior.

Will have a look tomorrow if you did not find a solution.

I created a custom php.ini and use it in the command:

php8.1 -c /data/project/mix-n-match/mixnmatch_rs/php.ini /data/project/mix-n-match/scripts/person_dates/update_person_dates.php 48922

The php.ini:

extension_dir="/layers/fagiani_apt/apt/usr/lib/php/20210902"

I also added a whole bunch of packages to the Aptfile, just in case.

No joy.

Thanks @taavi Is there a list of packages required for a toolforge-equivalent php setup?

For this the recommended way is using a php-only codebase, the issue is that you need both rust and php at the same time, and that's not so well supported (yet at least).

Looking

For my own testing, is there a way to get a shell into the docker container? I have seen the docs for the webservice one, but that doesn't seem to fly for non-webservice ones?

So, from within the container:

# Using podman, docker would be the same
dcaro@urcuchillay$ podman run --rm -ti --entrypoint launcher tools-harbor.wmcloud.org/tool-mix-n-match/tool-mix-n-match:latest bash

heroku@c5c28f3d0514:/workspace$ ls -la /layers/fagiani_apt/apt/usr/lib/php/20210902 | grep mysql
-rw-rw-rw-. 1 heroku heroku  162056 Jan  1  1980 mysqli.so
-rw-rw-rw-. 1 heroku heroku  231280 Jan  1  1980 mysqlnd.so
-rw-rw-rw-. 1 heroku heroku   35080 Jan  1  1980 pdo_mysql.so

heroku@c5c28f3d0514:/workspace$ cat php.ini 
extension_dir = "/layers/fagiani_apt/apt/usr/lib/php/20210902"

heroku@c5c28f3d0514:/workspace$ php8.1 -c php.ini  -i  | grep extension_
extension_dir => /layers/fagiani_apt/apt/usr/lib/php/20210902 => /layers/fagiani_apt/apt/usr/lib/php/20210902

## But
heroku@c5c28f3d0514:/workspace$ php8.1 -c php.ini  -i  | grep -i mysql
... nothing


heroku@c5c28f3d0514:/workspace$ php8.1 -c php.ini  -m  | grep -i mysql
... nothing

It feels more like a bug :/, looking

For my own testing, is there a way to get a shell into the docker container? I have seen the docs for the webservice one, but that doesn't seem to fly for non-webservice ones?

From your laptop you can pull the image, from toolforge, toolforge webservice buildservice shell should do the trick (there's a --buildservice-image if your image is not the default).

I have some updates. With php.ini as

extension_dir="/layers/fagiani_apt/apt/usr/lib/php/20210902"
extension=mysqlnd
extension=mysqli

it loads the mysql library. Still fails with some password error but that seems like something code-wise to debug.

In T359937#9625885, @Magnus wrote:

I have some updates. With php.ini as

extension_dir="/layers/fagiani_apt/apt/usr/lib/php/20210902"
extension=mysqlnd
extension=mysqli

it loads the mysql library. Still fails with some password error but that seems like something code-wise to debug.

Nice, yep, I just noted the same, when installing php in the system (with real apt install), it generates a default config file with those extra elements (and many others), that are not actually in the debian package, so probably generated after.

Small note that in order to make your code more future-proof, you might want to replace the hardcoded paths /data/project/mix-n-match with the path from the envvar $TOOL_DATA_DIR, not that it will change soon, but it might in the future.

@dcaro thanks, done