Requested in https://lists.wikimedia.org/pipermail/cloud/2019-January/000517.html
We may be able to make a smaller base image by making sure that we install local::lib and carton.
Our Docker containers tend to implement opinionated patterns for deploying and running code. It would be good to get feedback from actual Perl webservice authors on options for this. Examples:
i'd like to test somehow, which perl-modules are missing.
one answer in the mailing list was
webservice --backend=kubernetes TYPE shell will get you into a shell.
but that gave me just a message:
type must be one of golang,jdk8,nodejs,php5.6,php7.2,python,python2,ruby2,tcl
(however, i guess, i just need the debian package libhtml-template-perl for the perl module HTML::Template.)
We do not have any image at all for Perl yet, so that explains your failure with webservice --backend=kubernetes perl shell.
(however, i guess, i just need the debian package libhtml-template-perl for the perl module HTML::Template.)
Have you used any project local package manager (something that does not need root permissions) with your Perl projects in the past? I think our ideal solution would be something similar Python's venv or PHP's Composer which would allow each tool to install the 3rd party libraries it needs without us needing to place the union of all possible modules used by tools in the Perl Docker image itself. Ideally we would just be installing a core Perl runtime and the libraries/tools needed for local package management and then letting each tool maintainer install the extras their tool needs.
I've used CPAN locally at the old toolserver (ran by DaB.). Since there is labs, I just asked for some packages, and the admins installed them. For me this is an easy way.
In my opinion local package managing systems are a PITA. It's better to have centralized and centrally managed _system_ package bundles, where one can be sufficiently sure that everything works together.
All perl modules I use are packed for debian already, e.g.,
sudo apt-get install libcgi-pm-perl libdatetime-format-strptime-perl libhtml-template-perl libwww-perl
They are installed at the 'normal' labs server already.
I'm not used to work with docker images. I understand that those docker images should be small, because every tool get's its own image, right? So if it would be possible to install system packages in that container, that would be great, maybe something such as https://perlmaven.com/distributing-perl-script-using-docker?
Another possibility is to use lighttpd as on gridengine and just install missing modules like CGI.pm (created T250975 for that, but feel free to close as dupe)
Perl running webservices are still a blocker for gridengine to be replaced by kubernetes
The CheckWiki project is written in Perl 5. It already has a local library for addon Perl modules. It has both a webservice and grid jobs.
Change 778683 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):
[operations/docker-images/toollabs-images@master] Add perl532-sssd
Change 778683 merged by jenkins-bot:
[operations/docker-images/toollabs-images@master] Add perl532-sssd
Mentioned in SAL (#wikimedia-cloud) [2022-04-23T16:51:28Z] <bd808> Built new perl532-sssd/{base,web} images and pushed to registry (T214343)
Change 785374 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):
[operations/software/tools-webservice@master] k8s: add perl5.32 type
Change 785374 merged by jenkins-bot:
[operations/software/tools-webservice@master] k8s: add perl5.32 type
Change 785375 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):
[operations/software/tools-webservice@master] d/changelog: Prepare for 0.82 release
Change 785375 merged by jenkins-bot:
[operations/software/tools-webservice@master] d/changelog: Prepare for 0.82 release
Mentioned in SAL (#wikimedia-cloud) [2022-04-25T14:56:20Z] <bd808> Rebuilding all docker images to pick up toolforge-webservice v0.82 (T214343)
We did not come up with a good plan for self-service perl module management, but there is now a webservice --backend=kubernetes perl5.32 ... container which runs lighttpd with config to serve .pl files found in $HOME/public_html as CGI scripts. I believe this matches the general level of support we have for perl in webservice on the grid engine backend.
Change 789878 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):
[cloud/toolforge/jobs-framework-api@main] containers: Add perl 5.32 container
Change 789878 merged by jenkins-bot:
[cloud/toolforge/jobs-framework-api@main] containers: Add perl 5.32 container
We have never installed MediaWiki::Bot on the job grid, so I'm not sure why it would be required in a container.
perl532-sssd/base/Dockerfile.template has a list of the packages in the container. libmediawiki-bot-perl is listed.
become $TOOL; webservice perl5.32 shell will give you an interactive shell inside the perl container. From there you should be able to use perl directly and/or run scripts.
The checkwiki project has its extra modules installed in a project sub folder and uses a lib statement to add them to the search path.
Example: use lib '/data/project/checkwiki/perl/lib/perl5';
Yes, I could do that but it is not my preferred option because:
(1) The module is currently available on the grid
(2) It defeats the purpose of having a perl container
Of course, I handle my own modules this way.
using a use lib card is messy, as the path will not exist on the development machine, so better to include a path on the call line eg
perl -I File-Slurp-9999.32/lib bin/use.pl