Page MenuHomePhabricator
Create Task
Maniphest T307406

Python3 bot fails on new Buster tools grid
Closed, ResolvedPublic
Actions

Description

krinklebot fileprotectionsync.err:

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
0,10,20,40,50 * * * * /usr/bin/jsub -once -quiet -mem 500m -N fileprotectionsync $HOME/pywikienv/bin/python3 $HOME/src/pywiki-fileprotectionsync/fileprotectionsync.py

It fails when specifying -release buster to jsub.

See also:

Related Objects

Event Timeline

Krinkle created this task.May 2 2022, 11:41 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 2 2022, 11:41 PM
• taavi edited projects, added Toolforge; removed Cloud-VPS.May 3 2022, 7:15 AM
• taavi subscribed.

Hi @Krinkle! As far as I can tell, your jsub command references a virtualenv that was created for the Stretch grid (which has Python 3.5) available:

taavi@tools-sgebastion-11:/data/project/krinklebot/pywikienv/bin $ ls
activate  activate.csh  activate.fish  activate_this.py  chardetect  easy_install  easy_install-3.5  pip  pip3  pip3.5  python  python3  python3.5  python-config  wheel

To run your bot on the Buster grid, you'll likely need to re-create the virtualenv on a buster bastion as described here: https://wikitech.wikimedia.org/wiki/News/Toolforge_Stretch_deprecation#Rebuild_virtualenv_for_python_users

JeanFred mentioned this in T307269: ErfgoedBot doesn't work since 2022-04-19, since the move to new Buster tools grid.May 3 2022, 7:59 AM
Krinkle closed this task as Resolved.May 8 2022, 9:43 PM
Krinkle claimed this task.

Thanks @Majavah, that did it!

As part of re-installing this bot (readme) I decided to also update from pywikibot 6.6.1 to 7.2.0 while at it, and noticed that pywikibot-venv$ python setup.py develop no longer results in a working install, not even for a basic bot like mine that only instantiates Site and Page to save a plain text edit (no wikitext parsing). It simply unconditionally says that neither mwparserfromhell nor wikitextparser is installed, stating that it is a mandatory dependency now (ref T106763). I don't understand why why pywikibot can't have one of these as a default declared as a "real" dependency that the setup script always installs. Oh well, it works now!

Stashbot added a comment.May 8 2022, 9:45 PM

Mentioned in SAL (#wikimedia-cloud) [2022-05-08T21:45:54Z] <Krinkle> Migrate jsub to Buster grid, ref T307406

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits