Page MenuHomePhabricator

Upgrade nlwikibots to Stretch
Closed, ResolvedPublic

Description

nlwikibots runs:

  • lighttpd php webservice to redirect to the project site
    • trivial to convert
  • system Python 2.7.6 with sys.path mangling (older tools) / virtualenv (newer tools)
    • stretch runs 2.7.13, so should still be OK. Need to migrate to 3 at some point.
  • dplinks seems to be nonfunctional, so can probably be removed altogether
  • all bots return significant amounts of warnings from the API
    • probably good to bump the pywikibot version to something more recent

Event Timeline

I have changed the overall structure of the project directory a bit. There are now two shared virtualenvs:

  • ~/venv-py35-stretch
  • ~/venv-py27-stretch

both venvs have pywikibot and mwparserfromhell installed. The shared username & config are now stored in ~/.pywikibot, safe from prying eyes. This also means it's super easy to use pywikibot: simply drop a python script that uses pywikibot, and call

~/venv-py35-stretch/python <your-script>

and the script will automatically use the nlwikibots account.

I have converted tvpupdater to the new system, and have converterd archivering as well (quadruple upgrade: pywikibot compat -> core, python 2 -> python 3, trusty -> stretch and sys.path hacks -> virtualenv).

tvpmelder is still to do.

Took a look at tvpmelder -- this seems more of an headache:

  • complex code that uses both the api and the database
  • usage of both bytestrings and unicode strings due to the database access. Not clear whether this will work under Python 3
  • the AfD page layout has changed recently, and the bot doesn't seem to understands the new format
FrankGeerlings claimed this task.

Removed the last nlwikibots crontab entry (tvpmelder) from Trusty. Added tbpmelder (different code base, same general idea) to Stretch crontab.