Maniphest T320014

Migrate robokobot from Toolforge GridEngine to Toolforge Kubernetes
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	komla
	Oct 6 2022, 9:36 PM

Description

Kindly migrate your tool(https://grid-deprecation.toolforge.org/t/robokobot) from Toolforge GridEngine to Toolforge Kubernetes.

Toolforge GridEngine is getting deprecated.
See: https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/

Please note that a volunteer may perform this migration if this has not been done after some time.
If you have already migrated this tool, kindly mark this as resolved.

If you would rather shut down this tool, kindly do so and mark this as resolved.

Useful Resources:
Migrating Jobs from GridEngine to Kubernetes
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework#Grid_Engine_migration
Migrating Web Services from GridEngine to Kubernetes
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Move_a_grid_engine_webservice
Python
https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Rebuild_virtualenv_for_python_users

Related Objects

Mentioned Here: T355214: [apt-buildpack] Not sourcing /layers/fagiani_apt/apt/.profile.d/000_apt.sh
T356016: [builds-builder,jobs-api,upstream] Calling nontrivial Procfile commands with arguments results in confusing error (“no such file or directory”)
T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts
T314664: [infra] Decommission the Grid Engine infrastructure

Event Timeline

komla created this task.Oct 6 2022, 9:36 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 6 2022, 9:36 PM

My apologies if this ticket comes as a surprise to you. In order to ensure WMCS can provide a stable, secure and supported platform, it’s important we migrate away from GridEngine. I want to assure you that while it is WMCS’s intention to shutdown GridEngine as outlined in the blog post https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/, a shutdown date for GridEngine has not yet been set. The goal of the migration is to migrate as many tools as possible onto kubernetes and ensure as smooth a transition as possible for everyone. Once the majority of tools have migrated, discussion on a shutdown date is more appropriate. See T314664: [infra] Decommission the Grid Engine infrastructure.

As noted in https://techblog.wikimedia.org/2022/03/16/toolforge-gridengine-debian-10-buster-migration/ some use cases are already supported by kubernetes and should be migrated. If your tool can migrate, please do plan a migration. Reach out if you need help or find you are blocked by missing features. Most of all, WMCS is here to support you.

However, it’s possible your tool needs a mixed runtime environment or some other features that aren't yet present in https://techblog.wikimedia.org/2022/03/18/toolforge-jobs-framework/. We’d love to hear of this or any other blocking issues so we can work with you once a migration path is ready. Thanks for your hard work as volunteers and help in this migration!

Sorry I completely missed this ticket.

I tried following the instructions, but I can't seem to find the shared folder for pywikibot inside Kubernetes:
python3: can't open file '/shared/pywikibot/stable/pwb.py': [Errno 2] No such file or directory

My problem seems related to T249787.

I tried the toolforge-python39-sssd-pwb image but it doesn't work:

tools.robokobot@tools-sgebastion-10:~$ toolforge-jobs run virgule --command /data/project/robokobot/scripts/virgule2.sh --image toolforge-python39-sssd-pwb
ERROR: unable to create job: "HTTP 400: invalid container image"

In the meantime, I'm forced to use the Grid.
Please note that I use Toolforge not because it's free but because I don't have to worry about updating Pywikibot.

Thibaut120094 removed Thibaut120094 as the assignee of this task.Jan 20 2023, 2:05 PM

Thibaut120094 subscribed.

• nskaggs moved this task from Backlog to Pywikibot (T249787) on the Grid-Engine-to-K8s-Migration board.Jan 20 2023, 5:45 PM

@Thibaut120094 T249787 is now resolved!
The updated help page is here: https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_Pywikibot_scripts

Thank you, I set up the OAuth 1.0a environment variables and tried to run this command as a test:

toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb -family:wikipedia -lang:fr redirect double -always" fix-double-redirects

But the job keep failing after 5 seconds and I have no idea why because there are no logs, even when I try to force with -o or -e.

dcaro subscribed.Feb 14 2024, 11:02 AM

Hi @Thibaut120094!

I'm looking a bit into this. In order to get some logs I ran this:

tools.robokobot@tools-sgebastion-10:~$ toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --mount=all --command 'sh -c "exec 1>$TOOL_DATA_DIR/out; exec 2>$TOOL_DATA_DIR/err; pwb -family:wikipedia -lang:fr redirect double -always"' fix-double-redirects
## with that, it create err and out files
tools.robokobot@tools-sgebastion-10:~$ cat out
tools.robokobot@tools-sgebastion-10:~$ cat err
sh: 1: pwb: not found

Looking a bit more into the details, it seems it's affected by T356016: [builds-builder,jobs-api,upstream] Calling nontrivial Procfile commands with arguments results in confusing error (“no such file or directory”), looking

Okok, that should be fixed now, it was not that issue you were hitting, but a different one that was already fixed, but required us to rebuild the pywikibot image (the bug was T355214: [apt-buildpack] Not sourcing /layers/fagiani_apt/apt/.profile.d/000_apt.sh).

Just did that, and now it's able to find the pwb binary:

tools.robokobot@tools-sgebastion-10:~$ toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb -badoption" fix-double-redirects
...

tools.robokobot@tools-sgebastion-10:~$ toolforge jobs logs fix-double-redirects
2024-02-14T12:03:14+00:00 [fix-double-redirects-knmz2] family and mylang are not set.
2024-02-14T12:03:14+00:00 [fix-double-redirects-knmz2] Defaulting to family='wikipedia' and mylang='test'.
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2] WARNING: API error mwoauth-invalid-authorization: The authorization headers in your request are not valid: Invalid signature
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2] Traceback (most recent call last):
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pwb.py", line 39, in <module>
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     sys.exit(main())
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pwb.py", line 35, in main
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     runpy.run_path(str(path), run_name='__main__')
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 288, in run_path
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     return _run_module_code(code, init_globals, run_name,
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 97, in _run_module_code
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     _run_code(code, mod_globals, init_globals,
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/layers/heroku_python/python/lib/python3.9/runpy.py", line 87, in _run_code
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     exec(code, run_globals)
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/scripts/wrapper.py", line 513, in <module>
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     main()
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/scripts/wrapper.py", line 497, in main
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     if not execute():
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/scripts/wrapper.py", line 435, in execute
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     unknown_args = pwb.handle_args(global_args)
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/bot.py", line 1010, in handle_args
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     pywikibot.Site()
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/__init__.py", line 266, in Site
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     _sites[key] = interface(code=code, fam=fam, user=user)
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/site/_apisite.py", line 127, in __init__
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     self.login(cookie_only=True)
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/site/_apisite.py", line 396, in login
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     raise e
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/site/_apisite.py", line 387, in login
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     if self.userinfo['name'] == self.user():
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/site/_apisite.py", line 553, in userinfo
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     uidata = uirequest.submit()
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]   File "/workspace/pywikibot/data/api/_requests.py", line 1070, in submit
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2]     raise NoUsernameError(f'Failed {msg}')
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2] pywikibot.exceptions.NoUsernameError: Failed OAuth authentication for wikipedia:test: The authorization headers in your request are not valid: Invalid signature
2024-02-14T12:03:16+00:00 [fix-double-redirects-knmz2] CRITICAL: Exiting due to uncaught exception NoUsernameError: Failed OAuth authentication for wikipedia:test: The authorization headers in your request are not valid: Invalid signature

Can you retry now? (if the command you are trying is safe to run by me, I can try with too for testing if it does not work)

The grid engine has been shut down, so I'm closing any remaining migration tasks as Declined. If you're still planning to migrate this tool, please re-open this task and add one or more active project tags to it. (If you need a project tag for your tool, those can be created via the Toolforge admin console.)

Migrate robokobot from Toolforge GridEngine to Toolforge KubernetesClosed, DeclinedPublicActions

Description

Related Objects

Event Timeline

Migrate robokobot from Toolforge GridEngine to Toolforge Kubernetes
Closed, DeclinedPublic
Actions