Page MenuHomePhabricator

Python3 bot fails on new Buster tools grid
Closed, ResolvedPublic


krinklebot fileprotectionsync.err:

Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
0,10,20,40,50 * * * * /usr/bin/jsub -once -quiet -mem 500m -N fileprotectionsync $HOME/pywikienv/bin/python3 $HOME/src/pywiki-fileprotectionsync/

It fails when specifying -release buster to jsub.

See also:

Event Timeline

taavi added a subscriber: taavi.

Hi @Krinkle! As far as I can tell, your jsub command references a virtualenv that was created for the Stretch grid (which has Python 3.5) available:

taavi@tools-sgebastion-11:/data/project/krinklebot/pywikienv/bin $ ls
activate  activate.csh  chardetect  easy_install  easy_install-3.5  pip  pip3  pip3.5  python  python3  python3.5  python-config  wheel

To run your bot on the Buster grid, you'll likely need to re-create the virtualenv on a buster bastion as described here:

Krinkle claimed this task.

Thanks @Majavah, that did it!

As part of re-installing this bot (readme) I decided to also update from pywikibot 6.6.1 to 7.2.0 while at it, and noticed that pywikibot-venv$ python develop no longer results in a working install, not even for a basic bot like mine that only instantiates Site and Page to save a plain text edit (no wikitext parsing). It simply unconditionally says that neither mwparserfromhell nor wikitextparser is installed, stating that it is a mandatory dependency now (ref T106763). I don't understand why why pywikibot can't have one of these as a default declared as a "real" dependency that the setup script always installs. Oh well, it works now!

Mentioned in SAL (#wikimedia-cloud) [2022-05-08T21:45:54Z] <Krinkle> Migrate jsub to Buster grid, ref T307406