Pywikibot compat has more dependencies, many of which will become core dependencies as part of the ongoing OPW Pywikibot-compat-to-core project porting scripts to core. T71980 (over four months old) is raised for one of them.

Hmm, I'm wondering if we should figure out a virtualenv based way of setting up pywikibot on toollabs. This would give pywikibot maintainers the ability to maintain the required packages themselves without relying on the labs admins.

However, if the pywikibot folks think that maintaining the virtualenv themselves is going to be too much work / not something they want to do, I'm happy to install all these packages.

@yuvipanda, it would be great if you could install these dependencies in the first instance. Some of them are distributed as part of the OS package collection. Also I believe Pillow needs a sysadmin action, as Ubuntu puts python's Tkinter into a separate debian package.

If there are any packages which you dont feel comfortable installing globally, or if the dev team adds dependencies too frequently, we need to consider setting up maintained virtualenv package set, but hopefully we can avoid that. Most of these libraries are generally useful to anyone doing wiki stuff in python.

Please also add "socketIO_client" to the list; it is used to connect to the newish RecentChanges stream , so that it available to all python users.

Change 184298 had a related patch set uploaded (by Yuvipanda):
tools: Install some pywikibot dependencies

https://gerrit.wikimedia.org/r/184298

Patch-For-Review

Change 184298 merged by Yuvipanda:
tools: Install some pywikibot dependencies

https://gerrit.wikimedia.org/r/184298

So I've installed:

1. unittest2
2. pycountry
3. flickrapi
4. pydot

pysearch seems to be just pointing to python-yahoo now in ubuntu. Is that ok to be installed?

unicodecsv and socketIO require new packages to be built, so I shall do that later today.

Why are daemonize and crontab required?

daemonize supports the library command line option -daemonize. I dont
know much more than that, but I can dig into it.

crontab is used to parse a crontab stored on a wiki page, used to
schedule tasks. It is only used by a script that is not yet
functional, so not a high priority atm.

will look into the yahoo package

I'm not sure if we should support daemonize. Jobs should just go on the grid instead.

Change 184302 had a related patch set uploaded (by Yuvipanda):
tools: Install python-unicodecsv

https://gerrit.wikimedia.org/r/184302

Patch-For-Review

Change 184302 merged by Yuvipanda:
tools: Install python-unicodecsv

https://gerrit.wikimedia.org/r/184302

Change 184887 had a related patch set uploaded (by Yuvipanda):
tools: Install python-socketio-client

https://gerrit.wikimedia.org/r/184887

Patch-For-Review

Change 184887 merged by Yuvipanda:
tools: Install python-socketio-client

https://gerrit.wikimedia.org/r/184887

(sorry it took so long for me to get back to this)

1. python pysearch , I think I see what you are saying.

https://packages.debian.org/source/wheezy/python-pysearch is a transitional package for https://packages.debian.org/wheezy/python-yahoo

The filelist of https://packages.debian.org/wheezy/all/python-yahoo/filelist looks very similar to the source http://sourceforge.net/p/pysearch/code/HEAD/tree/trunk/yahoo/search/ listed at https://pypi.python.org/pypi/pYsearch , and it the appropriate .egg

So, I think only installing 'python-yahoo' is fine.

2. daemonize . Sorry, this can be ignored, as this external dependency is due to my misunderstanding; see T89581 .

p.s. If you dont want -daemonize used on tool labs, create a task to add a warning or error if it is used on tool labs. Tool labs is pywikibot's primary customer, so ... IMO the software should try to help enforce/promote the way you want the software used on your free hardware.

3. crontab

This module is only a crontab parser.

https://pypi.python.org/pypi/crontab
is
https://github.com/josiahcarlson/parse-crontab

It is used to parse the second item in each own of this page

https://de.wikipedia.org/wiki/Benutzer:DrTrigon/DrTrigonBot/script_wui-crontab.css

The first item is a revid, which is currently

https://de.wikipedia.org/wiki/?oldid=125193969

Those on-wiki 'files' are semi-protected due to the fact they are .css files.

The bot and operator was last active in August 2014 : https://de.wikipedia.org/wiki/Spezial:Beitr%C3%A4ge/DrTrigonBot / https://de.wikipedia.org/wiki/Spezial:Beitr%C3%A4ge/DrTrigon (https://tools.wmflabs.org/guc/?user=DrTrigon) https://gerrit.wikimedia.org/r/#/q/owner:%22DrTrigon+%253Cdr.trigon%2540surfeu.ch%253E%22,n,z They dont have a Phab account that I can see.

I would not mind if we deferred the 'what to do' with this script and its crontab dependency a little longer to see if DrTrigon becomes active. The dependency is optional, and nobody else is seriously working on improving this script at the moment (see T70797), so the dependency isn't needed for regression testing. The Lua dependency (mentioned in the task description) needs to be resolved before the crontab dependency is a serious impediment.

But wait, there is more!

Other dependencies added recently:

python3: six (probably already installed)
python2.6: future (hopefully you're not maintaining a py2.6 environment..;-)

Now that T76286 is merged, another dependency for py2.6 and 2.7 is https://pypi.python.org/pypi/ipaddress

Change 209978 had a related patch set uploaded (by Merlijn van Deen):
Add python-ipaddress package

https://gerrit.wikimedia.org/r/209978

Change 209978 abandoned by Merlijn van Deen:
Add python-ipaddress package

https://gerrit.wikimedia.org/r/209978

In T86015#1051029, @jayvdb wrote:

Now that T76286 is merged, another dependency for py2.6 and 2.7 is https://pypi.python.org/pypi/ipaddress

Now that T100603: Support Debian package python-ipaddr is resolved, that dependency for py2.7 has changed to Debian package python-ipaddr. Could it be installed please, and is it optional, but quite critical for performance.

Change 216904 had a related patch set uploaded (by Tim Landscheidt):
Tools: Add python-ipaddr to exec_environ

https://gerrit.wikimedia.org/r/216904

Change 216904 merged by Yuvipanda:
Tools: Add python-ipaddr to exec_environ

https://gerrit.wikimedia.org/r/216904

While Pillow is not yet installed, most of the core optional dependencies have been installed, and I think the desired approach is to create separate tasks and organise debian packages for them.