My apologies if this ticket comes as a surprise to you. In order to ensure WMCS can provide a stable, secure and supported platform, it’s important we migrate away from GridEngine. I want to assure you that while it is WMCS’s intention to shutdown GridEngine as outlined in the blog post https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/, a shutdown date for GridEngine has not yet been set. The goal of the migration is to migrate as many tools as possible onto kubernetes and ensure as smooth a transition as possible for everyone. Once the majority of tools have migrated, discussion on a shutdown date is more appropriate. See T314664: [infra] Decommission the Grid Engine infrastructure.

As noted in https://techblog.wikimedia.org/2022/03/16/toolforge-gridengine-debian-10-buster-migration/ some use cases are already supported by kubernetes and should be migrated. If your tool can migrate, please do plan a migration. Reach out if you need help or find you are blocked by missing features. Most of all, WMCS is here to support you.

However, it’s possible your tool needs a mixed runtime environment or some other features that aren't yet present in https://techblog.wikimedia.org/2022/03/18/toolforge-jobs-framework/. We’d love to hear of this or any other blocking issues so we can work with you once a migration path is ready. Thanks for your hard work as volunteers and help in this migration!

https://krdbot.toolforge.org/cgi-bin/Delinkerlog.pl

When I run via "webservice --backend=gridengine start" is works.
When I stop it and use ""webservice start" it breaks with "Can't locate CGI.pm".

https://krdbot.toolforge.org/cgi-bin/Denkmalgrep.pl

Also breaks on the gridengine with "loadable library and perl binaries are mismatched".

There is a perl5 directory in the tools.krdbot home, but I'm too old to understand what exactly I need to install locally for which platform. Any help appreciated.

In T319847#8299342, @Krd wrote:

https://krdbot.toolforge.org/cgi-bin/Delinkerlog.pl

When I run via "webservice --backend=gridengine start" is works.
When I stop it and use ""webservice start" it breaks with "Can't locate CGI.pm".

The default webservice type is currently php7.4. You can see this via webservice --help. For Perl support you would want the perl5.32 type: webservice perl5.32 start. That perl5.32 container should have all the same Perl modules installed that the legacy grid engine nodes have (T214343).

Making perl5.32 the default for your tool is possible by creating a service.template file. In your case it would probably look something like:

backend: kubernetes
type: perl5.32

When I do this I get version mismatches because the container runs with perl5.32 but the login host where I installed some additional perl modules has 5.28.

In T319847#8843551, @Krd wrote:

When I do this I get version mismatches because the container runs with perl5.32 but the login host where I installed some additional perl modules has 5.28.

You will need to use webservice perl5.32 shell to get a CLI environment where the perl version matches. This is functionally the same method that must be used for most runtimes (php, python, node, ...) as the containers and the bastions are not running the same software versions.

Sorry that I may sound stupid, but I have no experience how this all works.

tools.krdbot@tools-sgebastion-10:~$ webservice perl5.32 shell
type must be one of:
generic
  * lighttpd
  * lighttpd-plain
  * nodejs
  * uwsgi-plain
  * uwsgi-python
tools.krdbot@tools-sgebastion-10:~$

In T319847#8849577, @Krd wrote:

Sorry that I may sound stupid, but I have no experience how this all works.

I could have given better instructions. The list you are being shown is the available runtime types for --backend=gridengine. The default backend for webservice is --backend=kubernetes, but your tool currently has a $HOME/service.manifest file containing backend: gridengine which is taking precedent over the system default. You will need to be explicit about using Kubernetes while this is the case: webservice --backend=kubernetes perl5.32 shell

The shell now works, but I don't get all required the perl modules installed via CPAN for different reasons.

I think we shouldn't put much more time into this. If the gridengine instance can no longer run, please feel free remove it.

Hi @Krd, can you share your code on any public repo? (wikimedia gitlab/...), I can try giving it a go helping setup the environment.

Feel free to look into my code, but I don't like to publish that.

In T319847#9513361, @Krd wrote:

Feel free to look into my code, but I don't like to publish that.

I understand if you are reticent to publish the code, but any tool on toolforge should publish the code unless there's a strong, fully understood and acknowledged reason not to (ex. security issues).
And in order to use the build service (that would allow you to install custom packages https://wikitech.wikimedia.org/wiki/Help:Toolforge/Build_Service#Installing_Apt_packages), you need a public git url.

Asked and answered in May 2023: If the gridengine instance can no longer run, please feel free remove it.

This tool has been disabled from running on the Grid.

If you are the maintainer and you want this re-enabled so that you can work on migrating it off the grid, please reach out to the cloud admins on the mailing list(cloud-admin@lists.wikimedia.org)

However, please note that according to the timeline shared, on the 14th of March 2024, the grid infrastructure will be shut down.