Page MenuHomePhabricator

Missing Perl packages on dev.toolforge.org for anomiebot workflows
Open, HighPublicBUG REPORT

Description

Per your email, I find that libjson-perl is missing from the bastion. One of my scripts for submitting jobs for my tool uses this to parse JSON to know what needs submitting.

Another one of my scripts needs libdbi-perl:amd64 for database access.

Another needs libwww-perl to hit an HTTP-based API.

I suspect there will turn out to be more missing, but it's hard to tell since it fatals on the first missing package. Here's the full list of perl packages my code uses that seem to now be missing:

  • libbytes-random-secure-perl
  • libcrypt-gcrypt-perl
  • libdbi-perl:amd64
  • libdigest-crc-perl
  • libhtml-parser-perl
  • libhttp-message-perl
  • libjson-perl
  • libnet-oauth-perl
  • libpod-simple-wiki-perl
  • libredis-perl
  • liburi-perl
  • libwww-perl
  • libxml-libxml-perl

I don't know that I'd need _all_ of those on the bastion, but probably most of them.

Details

TitleReferenceAuthorSource BranchDest Branch
shell: wrap the shell in a launcher for buildservicesrepos/cloud/toolforge/tools-webservice!61dcaroadd_lanucher_to_shellmain
Customize query in GitLab

Event Timeline

@Anomie, can you run the scripts that need these Perl libraries from inside of a webservice perl5.32 shell container session? Or do they also need kubectl, toolforge-jobs, or similar software that is currently only available on the bastions?

bd808 renamed this task from Missing packages on dev.toolforge.org to Missing Perl packages on dev.toolforge.org for anomiebot workflows.Mar 20 2024, 12:13 AM

Something that is probably under advertised related to my webservice perl5.32 shell question is that webservice passes extra cli args into the container's shell which allows things like:

$ webservice perl5.32 shell -- perl -v

This is perl 5, version 32, subversion 1 (v5.32.1) built for x86_64-linux-gnu-thread-multi
(with 48 registered patches, see perl -V for more detail)

Copyright 1987-2021, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

This might be handy if there are things like shell scripts that need to run on the bastion which need some data from a script that needs to run in a container, assuming that data can be passed as captured command output from a subshell.

webservice perl5.32 shell

Just noting what we talked about, that "toolforge-shell" would be a lot more discoverable as a command name for that. 🙂

Or do they also need kubectl, toolforge-jobs, or similar software that is currently only available on the bastions?

I think I could work around the places that use kubectl in the same way I already do in some other places to hit the API directly when in a container. See also T321919 on that topic.

One script does need toolforge-jobs. I don't know what APIs would be needed to work around that one, or if they'd need further access reconfiguration.

Also, Perl has this neat thing where in my local dev environment I can configure the DBI as dbi:Gofer:transport=stream;url=ssh:login.toolforge.org;dsn=DBI:mysql:mysql_read_default_file=/home/anomie/replica.my.cnf;host=$WIKI.$SVC.db.svc.wikimedia.cloud;database=$WIKI_p and it'll ssh into the bastion to connect to the replica DB from there, instead of having to develop directly in toolforge or constantly sync code to be able to test it. Possibly I could set up my ~/.ssh/config with an entry that would use ProxyCommand to ssh→become→webservice shell→sshd -i though...

Possibly I could set up my ~/.ssh/config with an entry that would use ProxyCommand to ssh→become→webservice shell→sshd -i though...

First attempt using ProxyCommand seemed like it almost worked, except that needs to end with connecting to an sshd somewhere (either one already listening on a port or run as sshd -i) and there isn't an sshd on the exec node for it to connect to.

bd808 triaged this task as High priority.Apr 1 2024, 8:46 PM
bd808 added a project: cloud-services-team.
bd808 moved this task from Inbox to Needs discussion on the cloud-services-team board.

Dropping this into the "needs discussion" column for cloud-services-team as a blocker to decommissioning the remaining Buster bastions.

Dropping this into the "needs discussion" column for cloud-services-team as a blocker to decommissioning the remaining Buster bastions.

I see that, despite this, the main bastion at login.toolforge.org has been "updated" today and my scripts are now broken. 🙁

Dropping this into the "needs discussion" column for cloud-services-team as a blocker to decommissioning the remaining Buster bastions.

I see that, despite this, the main bastion at login.toolforge.org has been "updated" today and my scripts are now broken. 🙁

The announce email for switching login.toolforge.org to the new bastion style included a note that login-buster.toolforge.org can still be used by folks who need a "fat" bastion. The messaging could probably have been more proactive with an announce of the intention to change a day or so before actually changing to give more folks a chance to see the announcement, but time machines are in short supply at the moment.

I can't say that having to change various references to login.toolforge.org in my stuff to login-buster.toolforge.org (which seems like it will only work temporarily) seems like a very good solution.

It's pretty important that Anomie's stuff works. It would be great if it was considered a priority to help resolve this issue.

I have a mostly working solution for this issue in a custom container image created with the Toolforge build service, but it currently needs T356016: [builds-builder,jobs-api,upstream] Calling nontrivial Procfile commands with arguments results in confusing error (“no such file or directory”) to be fixed to make the container easier to use.

I have a mostly working solution for this issue in a custom container image created with the Toolforge build service, but it currently needs T356016: [builds-builder,jobs-api,upstream] Calling nontrivial Procfile commands with arguments results in confusing error (“no such file or directory”) to be fixed to make the container easier to use.

The shell script workaround does not help there?

I have a mostly working solution for this issue in a custom container image created with the Toolforge build service, but it currently needs T356016: [builds-builder,jobs-api,upstream] Calling nontrivial Procfile commands with arguments results in confusing error (“no such file or directory”) to be fixed to make the container easier to use.

The shell script workaround does not help there?

It might actually. I somehow missed that adding the shell script indirection then allows passing $@ through to the wrapped commands again. I will try making a generic wrapper and see if I can get that to do the needful.

@bd808 were you able to try it? I had a quick look at the container image repo, but it seems it's more than just one thing (some kind of templating system?) so I was not sure how to hook things in it.

@bd808 were you able to try it?

I have not yet made enough time to try adding wrapper scripts for the Procfile entrypoints. I did get far enough in reading to understand that what T356016#9542120 describes will require a separate launcher script for each entrypoint ("If you pass any other parameters, it will not wrap the command in a shell, but instead, try to execute it as one binary").

I had a quick look at the container image repo, but it seems it's more than just one thing (some kind of templating system?) so I was not sure how to hook things in it.

The src/... content is the Python launcher skeleton I have dreamed up for use with some other projects like https://gitlab.wikimedia.org/toolforge-repos/containers-bnc where I found a need to hack in a runtime setup step (usually generating a config file) to make things work as hoped. In this particular repo that code is not actually being used, but it was a convenient scaffold to hang the Python package installs off of.

My design goal is for this project is for the user to be able to interact with the container in such a way that it feels mostly like using the various toolforge ... and webservice commands on the bastion, but with the added benefit of having access to a Perl runtime with various libraries installed. The part I have been struggling with is actually getting things like toolforge jobs run to work within the container without requiring the user to use launcher in the command or to enter a special subshell with launcher bash. I am currently not sure if this goal of container transparency is actually possible with our available stack.

https://gitlab.wikimedia.org/toolforge-repos/bd808-buildpack-perl-bastion/-/merge_requests/1 is my work-in-progress feature branch. The branch uses Aptfile to install a number of utilities and Perl libraries. pyproject.toml is used to install Toolforge cli tools from their gitlab origin repos. Procfile tries to put various commands into the default $PATH, but fails to actually do this in a way that is functional for the Python scripts. This is where I thought that T356016: [builds-builder,jobs-api,upstream] Calling nontrivial Procfile commands with arguments results in confusing error (“no such file or directory”) was needed, but at this point I am not sure if that would actually help either. I probably need to make myself another feature branch that avoids T369563: fagiani/apt buildpack very slow when processing a large collection of packages so I can do some quicker iteration on potential solutions and thus better describe the problems I have encountered.