Page MenuHomePhabricator
Create Task
Maniphest T320062

Migrate steve-adder from Toolforge GridEngine to Toolforge Kubernetes
Closed, ResolvedPublic
Actions

Description

Kindly migrate your tool(https://grid-deprecation.toolforge.org/t/steve-adder) from Toolforge GridEngine to Toolforge Kubernetes.

Toolforge GridEngine is getting deprecated.
See: https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/

Please note that a volunteer may perform this migration if this has not been done after some time.
If you have already migrated this tool, kindly mark this as resolved.

If you would rather shut down this tool, kindly do so and mark this as resolved.

Useful Resources:
Migrating Jobs from GridEngine to Kubernetes
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework#Grid_Engine_migration
Migrating Web Services from GridEngine to Kubernetes
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Move_a_grid_engine_webservice
Python
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Rebuild_virtualenv_for_python_users

Related Objects
Search...

View Standalone Graph
This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.
StatusSubtypeAssignedTask
· · ·
ResolvedLegoktmT320062 Migrate steve-adder from Toolforge GridEngine to Toolforge Kubernetes
OpendcaroT194332 [Epic,builds-api,components-api,webservice,jobs-api] Make Toolforge a proper platform as a service with push-to-deploy and build packs
· · ·

Event Timeline

komla created this task.Oct 6 2022, 9:38 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 6 2022, 9:38 PM
• nskaggs subscribed.Oct 7 2022, 12:52 AM

My apologies if this ticket comes as a surprise to you. In order to ensure WMCS can provide a stable, secure and supported platform, it’s important we migrate away from GridEngine. I want to assure you that while it is WMCS’s intention to shutdown GridEngine as outlined in the blog post https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/, a shutdown date for GridEngine has not yet been set. The goal of the migration is to migrate as many tools as possible onto kubernetes and ensure as smooth a transition as possible for everyone. Once the majority of tools have migrated, discussion on a shutdown date is more appropriate. See T314664: [infra] Decommission the Grid Engine infrastructure.

As noted in https://techblog.wikimedia.org/2022/03/16/toolforge-gridengine-debian-10-buster-migration/ some use cases are already supported by kubernetes and should be migrated. If your tool can migrate, please do plan a migration. Reach out if you need help or find you are blocked by missing features. Most of all, WMCS is here to support you.

However, it’s possible your tool needs a mixed runtime environment or some other features that aren't yet present in https://techblog.wikimedia.org/2022/03/18/toolforge-jobs-framework/. We’d love to hear of this or any other blocking issues so we can work with you once a migration path is ready. Thanks for your hard work as volunteers and help in this migration!

Legoktm removed Legoktm as the assignee of this task.Oct 15 2022, 3:32 AM
Legoktm added a subtask: T194332: [Epic,builds-api,components-api,webservice,jobs-api] Make Toolforge a proper platform as a service with push-to-deploy and build packs.
Legoktm subscribed.
komla moved this task from Backlog to Needs custom image (T194332) on the Grid-Engine-to-K8s-Migration board.Oct 15 2022, 1:51 PM
Legoktm added a comment.EditedNov 5 2023, 5:30 AM

There's no container image that contains wget... guess I can use curl instead.

komla added a comment.Nov 6 2023, 5:03 AM

Can we say this is resolved?

Aklapper added a comment.Nov 10 2023, 11:04 AM

@komla Who is "we" and who are you addressing?

Legoktm closed this task as Resolved.Nov 17 2023, 7:07 AM
Legoktm claimed this task.

Probably me - I had merged the changes in Git but not yet deployed them. But now it's resolved!

tools.steve-adder@tools-sgebastion-10:~$ crontab -l
no crontab for tools.steve-adder
tools.steve-adder@tools-sgebastion-10:~$ toolforge jobs list
Job name:    Job type:         Status:
-----------  ----------------  --------------------------
download     schedule: @daily  Waiting for scheduled time
Izno subscribed.Nov 23 2023, 7:08 PM

https://steve-adder.toolforge.org/ seems to have gone down, I'd guess as a result of this task. :)

komla added a comment.Nov 30 2023, 3:09 PM
In T320062#9322023, @Aklapper wrote:

@komla Who is "we" and who are you addressing?

Yes, I was referring to Legoktm

Legoktm added a comment.Nov 30 2023, 5:01 PM
In T320062#9355901, @Izno wrote:

https://steve-adder.toolforge.org/ seems to have gone down, I'd guess as a result of this task. :)

Oops sorry, I messed up the deployment, its back now. And I will add health checks to automatically complain...

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits