My apologies if this ticket comes as a surprise to you. In order to ensure WMCS can provide a stable, secure and supported platform, it’s important we migrate away from GridEngine. I want to assure you that while it is WMCS’s intention to shutdown GridEngine as outlined in the blog post https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/, a shutdown date for GridEngine has not yet been set. The goal of the migration is to migrate as many tools as possible onto kubernetes and ensure as smooth a transition as possible for everyone. Once the majority of tools have migrated, discussion on a shutdown date is more appropriate. See T314664: [infra] Decommission the Grid Engine infrastructure.

As noted in https://techblog.wikimedia.org/2022/03/16/toolforge-gridengine-debian-10-buster-migration/ some use cases are already supported by kubernetes and should be migrated. If your tool can migrate, please do plan a migration. Reach out if you need help or find you are blocked by missing features. Most of all, WMCS is here to support you.

However, it’s possible your tool needs a mixed runtime environment or some other features that aren't yet present in https://techblog.wikimedia.org/2022/03/18/toolforge-jobs-framework/. We’d love to hear of this or any other blocking issues so we can work with you once a migration path is ready. Thanks for your hard work as volunteers and help in this migration!

I have migrated a cronjob for regular cleanups to toolforge-jobs. However I cannot migrate my backend, a perl script that communicates with the Java frontend using Redis. The script can run (/data/project/vcat/bin/gridserver) using the tf-perl532 image, but it does not work because Graphviz is not installed.

The required packages were installed globally for me back in 2013/2014 so they were available in GridEngine. I think I found the commits that contained the configuration changes to install Graphviz (https://phabricator.wikimedia.org/rOPUP8df13f97487044fc55e2583b1d15a34e5b7edbdf) and some fonts (https://phabricator.wikimedia.org/rOPUP6c1b6ec0c64c9d90b3618a9c6c7cf1e16353fa04#change-DJMMgi3f1EXd) I need.

I know the Dockerfiles for the images are in Git somewhere, so I could probably write a Dockerfile myself to create an image based on tf-perl543 with the packages I need. It is the simplest way I see to get this working, but I doubt the WMCS team wants to offer individual images for tools this way.

@Dapete Custom images for tools is a potential option in the future with the development of T194332: [Epic,builds-api,components-api,webservice,jobs-api] Make Toolforge a proper platform as a service with push-to-deploy and build packs. This work is still on-going, but it sounds like this might be the easiest migration path for you.

In T320118#8303084, @Dapete wrote:

I have migrated a cronjob for regular cleanups to toolforge-jobs. However I cannot migrate my backend, a perl script that communicates with the Java frontend using Redis. The script can run (/data/project/vcat/bin/gridserver) using the tf-perl532 image, but it does not work because Graphviz is not installed.

We are testing out buildpacks on Toolforge. Can you take a look to see if it can help you migrate using a custom image?
There's a short quickstart here.
Kindly take a look and let us know.

In T320118#8800128, @komla wrote:

We are testing out buildpacks on Toolforge. Can you take a look to see if it can help you migrate using a custom image?
There's a short quickstart here.
Kindly take a look and let us know.

I am leaving for a 4 week vacation today. I have set a reminder to look into this as soon as possible when I'm back home.