My apologies if this ticket comes as a surprise to you. In order to ensure WMCS can provide a stable, secure and supported platform, it’s important we migrate away from GridEngine. I want to assure you that while it is WMCS’s intention to shutdown GridEngine as outlined in the blog post https://techblog.wikimedia.org/2022/03/14/toolforge-and-grid-engine/, a shutdown date for GridEngine has not yet been set. The goal of the migration is to migrate as many tools as possible onto kubernetes and ensure as smooth a transition as possible for everyone. Once the majority of tools have migrated, discussion on a shutdown date is more appropriate. See T314664: [infra] Decommission the Grid Engine infrastructure.

As noted in https://techblog.wikimedia.org/2022/03/16/toolforge-gridengine-debian-10-buster-migration/ some use cases are already supported by kubernetes and should be migrated. If your tool can migrate, please do plan a migration. Reach out if you need help or find you are blocked by missing features. Most of all, WMCS is here to support you.

However, it’s possible your tool needs a mixed runtime environment or some other features that aren't yet present in https://techblog.wikimedia.org/2022/03/18/toolforge-jobs-framework/. We’d love to hear of this or any other blocking issues so we can work with you once a migration path is ready. Thanks for your hard work as volunteers and help in this migration!

The tool was successfully migrated to buster on April 9, 2022. It was discussed here: https://wikitech.wikimedia.org/w/index.php?title=Help_talk:Toolforge/Pywikibot&diff=1967612&oldid=1966965

Since then, there have been no changes in the settings, or the cron commands. I am not sure why the tool is showing up on grid engine again.

A few minutes ago, I updated my crontab command to the following format:

jsub -N mywrappedjob-stretch -release stretch ./my-wrapper.sh
jsub -N KiranBOT1 -release stretch kiranbot1.sh

previously it was:

jsub -release buster -N KiranBOT1 kiranbot1.sh

Even with the new command, it is still running on grid engine.

Yes, as mentioned in the other ticket, your tool is successfully running on
Buster grid.

We're encouraging maintainers to move to Kubernetes since Kubernetes is the
future.

In the initial announcement, we stated that Buster grid is a stop-gap
measure to moving all tools to Kubernetes.

I followed the directions given at:
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Kubernetes
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Python#Kubernetes_python_jobs
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Jobs_framework
https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot
and
https://en.wikipedia.org/wiki/User:Novem_Linguae/Essays/Toolforge_bot_tutorial

Still, the bot is not running:
https://k8s-status.toolforge.org/namespaces/tool-kiranbot4/

There is no clear tutorial/user-guide.

There are a few independencies/contradictions in above linked help pages.
For one, as directed at https://wikitech.wikimedia.org/wiki/Help:Toolforge/Python#Kubernetes_python_jobs
I created bootstrap_venv.sh with exact same content as provided. After running the "toolforge-jobs run bootstrap-venv --command "cd $PWD && ./bootstrap_venv.sh" --image tf-python39 --wait" command, I get:
INFO: job 'bootstrap_venv' completed

In cronjobs.yaml I have following content:

- name: exp1
  command: ./task1.sh
  image: tf-python39
  no-filelog: false
  schedule: "50 13 * * *"
  emails: onfailure

In exp1.err I have following content:

Traceback (most recent call last):
  File "/data/project/shared/pywikibot/stable/scripts/replace.py", line 155, in <module>
    import pywikibot
  File "/data/project/shared/pywikibot/stable/pywikibot/__init__.py", line 43, in <module>
    from pywikibot.bot import (
  File "/data/project/shared/pywikibot/stable/pywikibot/bot.py", line 111, in <module>
    from pywikibot import config, daemonize, i18n, version
  File "/data/project/shared/pywikibot/stable/pywikibot/version.py", line 26, in <module>
    from pywikibot.comms.http import fetch
  File "/data/project/shared/pywikibot/stable/pywikibot/comms/http.py", line 40, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

Any suggestions regarding what am I doing wrong, and what should I do next?

tomorrow I will test the following file from https://en.wikipedia.org/wiki/User:Novem_Linguae/Essays/Toolforge_bot_tutorial#Example_cronjobs.yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  name: task1
  labels:
    name: kiranbot4.task1
    # The toolforge=tool label will cause $HOME and other paths to be mounted from Toolforge
    toolforge: tool
spec:
  schedule: "0 9 * * *"  # 9th hour UTC, 2:30PM IST.
  jobTemplate:
    spec:
      template:
        metadata:
          labels:
            toolforge: tool
        spec:
          containers:
          - name: bot
            workingDir: /data/project/kiranbot4
            image: docker-registry.tools.wmflabs.org/toolforge-python39-sssd-base:latest
            args:
            - /bin/bash
            - -c
            - /data/project/kiranbot4/task1.sh
            env:
            - name: HOME
              value: /data/project/kiranbot4
          restartPolicy: Never

I will update the results here when I do it.

Apparently, there are some modules missing in the "shared Pywikibot files (recommended setup)", and you guys are not responding other than rushing tool operators to migrate while back-end (from your side) is not completely ready yet.

there are some modules missing in the "shared Pywikibot files (recommended setup)"

"some modules" is not helpful. All of the required modules should be present.

In T319844#8322260, @Usernamekiran wrote:

Apparently, there are some modules missing in the "shared Pywikibot files (recommended setup)", and you guys are not responding other than rushing tool operators to migrate while back-end (from your side) is not completely ready yet.

T249787: Create Docker image for Toolforge that is purpose built to run pywikibot scripts may be what you really are hoping for @Usernamekiran. The "recommended" https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot#Using_the_shared_Pywikibot_files_(recommended_setup) instructions are very much grid engine focused. Using some variation on the https://wikitech.wikimedia.org/wiki/Help:Toolforge/Pywikibot#Setup_pywikibot_on_Toolforge_(locally) instructions with Python 3.9 Kubernetes runtime is currently a better fit for Kubernetes users.

The Python Kubernetes containers generally do not install Python library packages like requests at a system level. Python libraries are instead expected to be installed in virtual environments managed by each individual tool.

I have successfully migrated it. In 3-4 days, I will create a user-guide for other users. I apologise for my previous comments if they sounded passive-aggressive. I did not meant them to be that way, but after reading them later, I realised they may look like it. Thank you everybody for your help/guidance. See you around.

Thank you!