Page MenuHomePhabricator

itwiki jobs still running on Stretch Grid Engine
Closed, ResolvedPublic

Description

The web service (of the deprecated type php5.6) at https://itwiki.toolforge.org runs on Kubernetes already, but several recurring jobs seem to be scheduled on the older infrastructure (mainly via crontab). Those should be migrated to either Kubernetes (preferred) or the Buster Grid Engine.
Please see: https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation
Also: itwiki jobs on Stretch grid engine in last 7 days

Tools needing migration to Kubernetes:

  • webservice (was: php 5.6; now: php7.2)
  • archivebot (was: grid)
  • wikiquotestats
  • cleansandbox
  • datiwikipedie
  • deletionbot (was: grid PHP 7.2.31; now: moved to Kubernetes PHP 7.4)
  • markadmins (was: grid PHP 7.2.31; now: moved to Kubernetes PHP 7.4)
  • orphanizerbot (was: grid PHP 7.2.31; now: moved from to Kubernetes PHP 7.4)
  • draftbot
  • draftbot/listdrafts
  • lavorosporco

Details

Other Assignee
Sakretsu

Event Timeline

valerio.bozzolan subscribed.

Yup thank you. (I received 7 emails this week-end about this).

The webserver itself was migrated to Kubernetes with:

ssh valeriobozzolan@login-buster.toolforge.org
become itwiki
webservice stop
webservice --backend=kubernetes php7.2 start

I handled the tasks in Python, hopefully everything works fine.

I've migrated my children to Kubernetes. Here some notes.

Script markadmins

From:

5 * * * * jsub -once -N itwiki-markadmins -o $HOME/markadmins/log.out -e $HOME/markadmins/log.err $HOME/markadmins/bot.php >/dev/null 2>&1

To:

toolforge-jobs run itwiki-markadmins --command ./markadmins/bot.php --image tf-php74 --schedule "5 * * * *"

Script orphanizerbot

From:

*/3 * * * * /usr/bin/jsub -N cron-52 -once -quiet $HOME/orphanizerbot/start-in-toolforge.sh
orphanizerbot/start-in-toolforge.sh
...
./orphanizerbot/orphanize.php \
--warmup=120 \
--delay=9 \
--cooldown=50 \
--no-interaction

To:

toolforge-jobs run itwiki-orphanizerbot --command "./orphanizerbot/orphanize.php --warmup=120 --delay=9 --cooldown=50 --no-interaction" --image tf-php74 --schedule "*/3 * * * *"

Script deletionbot

From:

*/8 * * * * /usr/bin/jsub -N cron-37 -once -quiet $HOME/deletionbot/start-in-toolforge.sh
deletionbot/start-in-toolforge.sh
...
./deletionbot/bot.php --days=11

To:

toolforge-jobs run itwiki-deletionbot --command "./deletionbot/bot.php --days=11" --image tf-php74 --schedule "*/8 * * * *"

For now it seems everything is OK and with a very simple and smooth migration, without any minimal trouble of any kind. Wow.

Let's see tomorrow morning if everything is on fire.

It's not clear to me if the toolforge-jobs command in its automagic features also persists the --schedule also after a reboot of the tools-sgebastion-10 itself or not. I'm positive BTW.

valerio.bozzolan updated the task description. (Show Details)
valerio.bozzolan moved this task from Backlog to Toolforge on the Tool-itwiki board.

It's not clear to me if the toolforge-jobs command in its automagic features also persists the --schedule also after a reboot of the tools-sgebastion-10 itself or not. I'm positive BTW.

It functions independently of the bastion. You can see the stored jobs, including the schedule, with toolforge-jobs list (add --long for more info).

OK so that's a view of the stored jobs not a view of the jobs planned in the current "session or whatever temporary". Thank you. That's awesome!

Everything runs smoothly. The provided Kubernetes interface is just super-simple and awesome.

Thank you for your report @Ricordisamoa!