Page MenuHomePhabricator

Upgrade nlwikibots to Stretch
Closed, ResolvedPublic

Description

nlwikibots runs:

  • lighttpd php webservice to redirect to the project site
    • trivial to convert
  • system Python 2.7.6 with sys.path mangling (older tools) / virtualenv (newer tools)
    • stretch runs 2.7.13, so should still be OK. Need to migrate to 3 at some point.
  • dplinks seems to be nonfunctional, so can probably be removed altogether
  • all bots return significant amounts of warnings from the API
    • probably good to bump the pywikibot version to something more recent

Event Timeline

Restricted Application added subscribers: tools-nlwikibots-list, Aklapper. · View Herald TranscriptFeb 17 2019, 1:48 PM

I have changed the overall structure of the project directory a bit. There are now two shared virtualenvs:

  • ~/venv-py35-stretch
  • ~/venv-py27-stretch

both venvs have pywikibot and mwparserfromhell installed. The shared username & config are now stored in ~/.pywikibot, safe from prying eyes. This also means it's super easy to use pywikibot: simply drop a python script that uses pywikibot, and call

~/venv-py35-stretch/python <your-script>

and the script will automatically use the nlwikibots account.

I have converted tvpupdater to the new system, and have converterd archivering as well (quadruple upgrade: pywikibot compat -> core, python 2 -> python 3, trusty -> stretch and sys.path hacks -> virtualenv).

tvpmelder is still to do.

Took a look at tvpmelder -- this seems more of an headache:

  • complex code that uses both the api and the database
  • usage of both bytestrings and unicode strings due to the database access. Not clear whether this will work under Python 3
  • the AfD page layout has changed recently, and the bot doesn't seem to understands the new format
FrankGeerlings closed this task as Resolved.Mar 19 2019, 1:48 AM
FrankGeerlings claimed this task.

Removed the last nlwikibots crontab entry (tvpmelder) from Trusty. Added tbpmelder (different code base, same general idea) to Stretch crontab.