My apologies if this ticket comes as a surprise to you. In order to ensure WMCS can provide a stable, secure and supported platform, it’s important we migrate away from GridEngine. I want to assure you that while it is WMCS’s intention to shutdown GridEngine as outlined in the blog post https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/, a shutdown date for GridEngine has not yet been set. The goal of the migration is to migrate as many tools as possible onto kubernetes and ensure as smooth a transition as possible for everyone. Once the majority of tools have migrated, discussion on a shutdown date is more appropriate. See T314664: [infra] Decommission the Grid Engine infrastructure.

As noted in https://techblog.wikimedia.org/2022/03/16/toolforge-gridengine-debian-10-buster-migration/ some use cases are already supported by kubernetes and should be migrated. If your tool can migrate, please do plan a migration. Reach out if you need help or find you are blocked by missing features. Most of all, WMCS is here to support you.

However, it’s possible your tool needs a mixed runtime environment or some other features that aren't yet present in https://techblog.wikimedia.org/2022/03/18/toolforge-jobs-framework/. We’d love to hear of this or any other blocking issues so we can work with you once a migration path is ready. Thanks for your hard work as volunteers and help in this migration!

Hello,

I would like to explain what is this being used for right now and confirm that shutdown of this service is not critical for Huggle operation:

There are historically 2 external locations (web services) that Huggle clients are using:

• Whitelist server - which was migrated outside of labs some time ago, now running on my private VPS (https://huggle.bena.rocks)
• Updater server - which I believe is still here on grid engine, although I am not 100% sure. I just checked the source code - https://github.com/huggle/huggle3-qt-lx/blob/master/src/huggle_ui/updateform.cpp#L86 which contains:

this->qData->URL = "https://huggle.toolforge.org/updater/?version=" + QUrl::toPercentEncoding(HUGGLE_VERSION)
        + "&os=" + QUrl::toPercentEncoding(Configuration::HuggleConfiguration->Platform)
        + "&language=" + Localizations::HuggleLocalizations->PreferredLanguage;

This used to be hosted as a tools "web service" that was managed using helper scripts from Marc-Andre, I don't know if this was transparently migrated to k8s, but I doubt that. It's rather easy to deploy updater scripts somewhere else, but changing the URL is non-trivial since it seems to be hardcoded into the application itself.

With updater URL being defunct, Huggle is only going to lose the ability to inform its users that there is a new version. That isn't ideal, but it isn't breaking either.

I found this related merge request in Huggle repo - https://github.com/huggle/huggle3-qt-lx/pull/351 where @Se4598 updated the updater URL from / to

this->qData->URL = "http://tools.wmflabs.org/huggle/updater/?version=" + QUrl::toPercentEncoding(HUGGLE_VERSION)
this->qData->URL = "https://huggle.toolforge.org/updater/?version=" + QUrl::toPercentEncoding(HUGGLE_VERSION)

So maybe this indeed is already migrated to k8s? Can you maybe shed some light on this?

@Petrb Hi! That's just the current url of the webservice, it's the same for kubernetes and for grid webservices (https://<toolname>.toolforge.org), I can see that there's still a lighthttpd webservice running on the Grid:

huggle
members:

    addshore petrb se4598 

	cron-tools.huggle-1 	1008 	2024-02-13 14:20
lighttpd-huggle 	1 	Currently running

From https://grid-deprecation.toolforge.org/t/huggle

@dcaro ok but can we somehow check if this URL https://huggle.toolforge.org/ is grid or k8s?

I think grid was this URL - http://tools.wmflabs.org/huggle/ which we no longer use

In T319797#9540970, @Petrb wrote:

@dcaro ok but can we somehow check if this URL https://huggle.toolforge.org/ is grid or k8s?

I think grid was this URL - http://tools.wmflabs.org/huggle/ which we no longer use

Yes, it is being served from the grid (the lighttpd process showing up in https://grid-deprecation.toolforge.org/t/huggle), you can also check by running:

tools.huggle@tools-sgebastion-10:~$ toolforge webservice status
Your webservice of type lighttpd is running on backend gridengine

Both urls redirect to the same place (the *.toolforge.org is the new one, the tools.wmflabs.org/* is the deprecated version, only old tools have those), and from the url you can't tell if it's the grid or k8s.

Once you start the webservice on k8s, that same url will automatically point to the k8s process.

I am trying to follow instructions here - https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Move_a_grid_engine_webservice

but I am getting this error trying to stop the current webservice

tools.huggle@tools-sgebastion-10:~$ webservice stop
--mount is only for --backend=kubernetes
Please run again without this argument

also this

tools.huggle@tools-sgebastion-10:~$ webservice --backend=kubernetes php7.2 start
Backend 'gridengine' from /data/project/huggle/service.manifest does not match 'kubernetes'
  Try stopping your current webservice:
    webservice stop
  Then try starting it again:
    /usr/local/bin/webservice --backend=kubernetes php7.2 start
  If you have already tried that and it did not help, remove the state file before retrying:
    rm /data/project/huggle/service.manifest

I think the instructions are wrong.

In T319797#9593193, @Petrb wrote:

tools.huggle@tools-sgebastion-10:~$ toolforge webservice --backend=gridengine stop
--mount is only for --backend=kubernetes
Please run again without this argument

I can reproduce, looking

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/tools-webservice/-/merge_requests/29

cli: don't require mount for gridengine

dcaro merged https://gitlab.wikimedia.org/repos/cloud/toolforge/tools-webservice/-/merge_requests/30

d/changelog: bump to 0.103.4

Thanks for your patience, deployed a fix and tested, can you verify that it works for you now?

yes it works now