Page MenuHomePhabricator

Migrate ruprecht from Toolforge GridEngine to Toolforge Kubernetes
Closed, DeclinedPublic

Description

Kindly migrate your tool(https://grid-deprecation.toolforge.org/t/ruprecht) from Toolforge GridEngine to Toolforge Kubernetes.

Toolforge GridEngine is getting deprecated.
See: https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/

Please note that a volunteer may perform this migration if this has not been done after some time.
If you have already migrated this tool, kindly mark this as resolved.

If you would rather shut down this tool, kindly do so and mark this as resolved.

Useful Resources:
Migrating Jobs from GridEngine to Kubernetes
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework#Grid_Engine_migration
Migrating Web Services from GridEngine to Kubernetes
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Move_a_grid_engine_webservice
Python
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Rebuild_virtualenv_for_python_users

Event Timeline

My apologies if this ticket comes as a surprise to you. In order to ensure WMCS can provide a stable, secure and supported platform, it’s important we migrate away from GridEngine. I want to assure you that while it is WMCS’s intention to shutdown GridEngine as outlined in the blog post https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/, a shutdown date for GridEngine has not yet been set. The goal of the migration is to migrate as many tools as possible onto kubernetes and ensure as smooth a transition as possible for everyone. Once the majority of tools have migrated, discussion on a shutdown date is more appropriate. See T314664: [infra] Decommission the Grid Engine infrastructure.

As noted in https://techblog.wikimedia.org/2022/03/16/toolforge-gridengine-debian-10-buster-migration/ some use cases are already supported by kubernetes and should be migrated. If your tool can migrate, please do plan a migration. Reach out if you need help or find you are blocked by missing features. Most of all, WMCS is here to support you.

However, it’s possible your tool needs a mixed runtime environment or some other features that aren't yet present in https://techblog.wikimedia.org/2022/03/18/toolforge-jobs-framework/. We’d love to hear of this or any other blocking issues so we can work with you once a migration path is ready. Thanks for your hard work as volunteers and help in this migration!

Boldly adding MediaWiki-Platform-Team to open Ruprecht tasks as https://toolsadmin.wikimedia.org/tools/id/ruprecht does not provide a dedicated issue tracking location, as https://gerrit.wikimedia.org/g/mediawiki/tools/dependency-analysis provides no hints where to track issues, and as the Core Platform Team Initiatives (Decoupling (CDP2)) team tag is archived and its parent Platform Engineering team does not exist anymore. (Please strongly consider the good practice to have a dedicated *codebase* project tag for Ruprecht, as teams (and their tags) may cease to exist while codebases continue to exist, to avoid such bugmail noise in the future.)

We'll want a fresh start on dependency analysis. The Ruprecht projet was an early proof of concept.

taavi subscribed.

The ruprecht tool is still running on the grid engine. If it's no longer used, please stop the grid web services and/or properly delete the tool.

The ruprecht tool is still running on the grid engine. If it's no longer used, please stop the grid web services and/or properly delete the tool.

Hm, I would still like to be able to run it manually. I'm fine with stopping all the processes, but I would stil like to be able to log in...

The tool's web service seems to be just serving some static HTML and other files, so it should be relatively simple to migrate that to Kubernetes with something like:

$ webservice stop
$ webservice --backend=kubernetes php8.2 start

Migrating the generation scripts might b a bit more tricker, since those seem to mix php and python2/3 in the same scripts. But that only needs to migrated if you still need to run the scripts on Toolforge.

The ruprecht tool is still running on the grid engine. If it's no longer used, please stop the grid web services and/or properly delete the tool.

While I'd like to be able to still run the tool on toolforge, it looks like it would need quite a bit of work to migrate it to k8s. So I think it's best to just delete if from Toolforge.

However, the instructions seem to be outdated. The page sais that Maintainers can mark their tools for deletion using the "Disable tool" button on the tool's detail page on https://toolsadmin.wikimedia.org/. However, when I got to https://toolsadmin.wikimedia.org/tools/id/ruprecht, I get a "delete" button. When I click it, it sais "Are you sure you want to delete this toolinfo record?" - it says nothing about stopping services and such. Is this what I want? Or will I just be removing the record on toolinfo, and leave everythign running?

You're looking at the wrong button. The correct one is this one at the bottom of the left sidebar:

image.png (555×531 px, 27 KB)

You're looking at the wrong button. The correct one is this one at the bottom of the left sidebar:

Ah, thank you! In a narrow window, the side bar moves to the bottom, so the button was scroleld out of view... Silly me :)

I disabled the tool.