pywikibot.org landing page is not updated
Closed, DuplicatePublic
Actions

Assigned To

None

Authored By

	Xqt
	Jun 9 2025, 10:24 AM

Description

The pywikibot.org landing page or its target https://pywikibot.toolforge.org/ is not updated automatically anymore. The last update was on 28th March.

The repository can be found here. The nightly script is normaly executed twice a day but looking at the version information which was updated two weeks ago on 24th May the web page is outdated.

The last nichtly steps were:

Requirement already satisfied: pip in ./.kube_venv/lib/python3.11/site-packages (23.0.1)
Collecting pip
  Using cached pip-25.1.1-py3-none-any.whl (1.8 MB)
Requirement already satisfied: setuptools in ./.kube_venv/lib/python3.11/site-packages (66.1.1)
Collecting setuptools
  Using cached setuptools-80.9.0-py3-none-any.whl (1.2 MB)
Installing collected packages: setuptools, pip
  Attempting uninstall: setuptools
    Found existing installation: setuptools 66.1.1
    Uninstalling setuptools-66.1.1:
      Successfully uninstalled setuptools-66.1.1
  Attempting uninstall: pip
    Found existing installation: pip 23.0.1
    Uninstalling pip-23.0.1:
      Successfully uninstalled pip-23.0.1
Successfully installed pip-25.1.1 setuptools-80.9.0
Building new public_html in /tmp/tmp.HLCtBkvhsB on nightly-29157180-g5cdk...
Step 1: Collecting mwparserfromhell...
Step 2: Collecting setuptools...
Step 3: Collecting importlib_metadata for Python 3.7...
Step 4: Collecting typing_extensions for importlib_metadata...
Step 5: Collecting zipp for importlib_metadata...
Step 6: Build core master
Step 7: Build core stable
Step 8: zip ALL the things
Step 9: tar ALL the things
Step 10: Copy static files
Step 11: Remove old stuff
Cleaned up /tmp/tmp.HLCtBkvhsB

The err log gives

...
/data/project/pywikibot/.kube_venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py:289: UserWarning: Unknown distribution option: 'tests_require'
  warnings.warn(msg)
/data/project/pywikibot/.kube_venv/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: MIT License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()
/data/project/pywikibot/.kube_venv/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: Apache Software License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()
ModuleNotFoundError while trying to load entry-point bdist_wheel: No module named 'setuptools.command.bdist_wheel'
/data/project/pywikibot/.kube_venv/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: MIT License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()

rm: cannot remove 'public_html.old/core/pywikibot/__pycache__/version.cpython-311.pyc': Permission denied
...
rm: cannot remove 'public_html.old/core_stable/pkg_resources/extern/__pycache__/__init__.cpython-39.pyc': Permission denied

Related Objects
Search...

Status	Subtype	Assigned	Task
Open	Release	None	T396368 Pywikibot 11 release
Open	BUG REPORT	None	T396356 Creating the nightly package or a new package gives a bulk of warnings
Duplicate		None	T396338 pywikibot.org landing page is not updated

Event Timeline

Xqt created this task.Jun 9 2025, 10:24 AM

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptJun 9 2025, 10:24 AM

Xqt updated the task description. (Show Details)Jun 9 2025, 11:43 AM

It fails with the first statement of nightly batch:

# delete the venv, if it already exists
rm -rf .kube_venv

The error log is

/mnt/nfs/labstore-secondary-tools-project/pywikibot$ ./nightly-source/nightly
rm: cannot remove '.kube_venv/bin/python3.11': Permission denied
...
rm: cannot remove '.kube_venv/lib/python3.11/site-packages/setuptools-80.9.0.dist-info/top_level.txt': Permission denied
rm: cannot remove '.kube_venv/pyvenv.cfg': Permission denied
rm: cannot remove '.kube_venv/include/python3.11': Permission denied
rm: cannot remove '.kube_venv/lib64': Permission denied

Xqt added projects: SRE, cloud-services-team, Toolforge.Jun 9 2025, 12:27 PM

Xqt updated the task description. (Show Details)

taavi removed a project: SRE.Jun 9 2025, 12:34 PM

How is that command being run?

There is a crontab:

---
- name: pre-nightly
  command: ./pre_nightly
  image: bookworm
  schedule: "0 23,11 * * *"
#  mem: 1Gi
  emails: onfailure
- name: nightly
  command: ./nightly-source/nightly
  image: python3.11
  schedule: "0 1,13 * * *"
#  mem: 1Gi
  emails: onfailure

I tried to run ./nightly-source/nightly manually and got the last errors.

I re-started the nightly job manually with:

tools.pywikibot@tools-bastion-12:~$ toolforge jobs restart nightly

It completed successfully:

tools.pywikibot@tools-bastion-12:~$ kubectl get pod nightly-1749473193-s89q6
NAME                       READY   STATUS      RESTARTS   AGE
nightly-1749473193-s89q6   0/1     Completed   0          3m11s

And I don't see any errors in the logs either:

tools.pywikibot@tools-bastion-12:~$ cat nightly.{out,err}
Requirement already satisfied: pip in ./.kube_venv/lib/python3.11/site-packages (23.0.1)
Collecting pip
  Using cached pip-25.1.1-py3-none-any.whl (1.8 MB)
Requirement already satisfied: setuptools in ./.kube_venv/lib/python3.11/site-packages (66.1.1)
Collecting setuptools
  Using cached setuptools-80.9.0-py3-none-any.whl (1.2 MB)
Installing collected packages: setuptools, pip
  Attempting uninstall: setuptools
    Found existing installation: setuptools 66.1.1
    Uninstalling setuptools-66.1.1:
      Successfully uninstalled setuptools-66.1.1
  Attempting uninstall: pip
    Found existing installation: pip 23.0.1
    Uninstalling pip-23.0.1:
      Successfully uninstalled pip-23.0.1
Successfully installed pip-25.1.1 setuptools-80.9.0
Building new public_html in /tmp/tmp.QnpKDH735D on nightly-1749473193-s89q6...
Step 1: Collecting mwparserfromhell...
Step 2: Collecting setuptools...
Step 3: Collecting importlib_metadata for Python 3.7...
Step 4: Collecting typing_extensions for importlib_metadata...
Step 5: Collecting zipp for importlib_metadata...
Step 6: Build core master
Step 7: Build core stable
Step 8: zip ALL the things
Step 9: tar ALL the things
Step 10: Copy static files
Step 11: Remove old stuff
Step 12: Create public_html.new from temp
Step 13: Do the magic switcharoo
Cleaned up /tmp/tmp.QnpKDH735D
/data/project/pywikibot/.kube_venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py:289: UserWarning: Unknown distribution option: 'tests_require'
  warnings.warn(msg)
/data/project/pywikibot/.kube_venv/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: MIT License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()
/data/project/pywikibot/.kube_venv/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: Apache Software License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()
ModuleNotFoundError while trying to load entry-point bdist_wheel: No module named 'setuptools.command.bdist_wheel'
/data/project/pywikibot/.kube_venv/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: MIT License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()

So unless this starts happening again when the job runs I'm afraid there's nothing that the Toolforge admins can do here.

I had removed the error logs because they were very huge. There was another problem with deleting the public_html.old folder. @taavi: could you start a second run to verify if the old folder is deleted then. Previously we had to rename them in such case and there ar old folder which cannot be removed: