Added because they appear to be the primary developer/maintainer of the tool, and they may know the cause/source of the error.

@scfc No worries, but http://tools.wmflabs.org/spiarticleanalyzer/ seems to
work fine for me... (perhaps you fixed it?). By the way, the error doesn't
show up all the time, but shows up from time to time. Thought it worthwhile
to at least mention it on this task.

Commit referenced above abandoned. Task resolved.

webservice --backend=kubernetes python2 shell
python
import icu + press Enter yields:

Similar to T156626. Reinitiated discussion there. Case status stalled per associated ticket.

webservice stop in ~ ((venv)tools.spiarticleanalyzer@tools-bastion-03:~$) still yields

@bd808 Thanks for merging. Do(es) particular container(s) need to be rebuilt now? (discussion of next steps)

@yuvipanda Since the gerrit patch above probably won't necessarily go live when it is merged, how are we planning to rebuild icu-requesting containers?

In T157744#3025326, @tom29739 wrote:

PyICU requires the underlying C++ library to work, and this is in the libicu52 apt package on Debian Jessie.

@scfc webservice --backend=kubernetes python2 start (kubernetes), though, yields the output posted earlier in the ticket (ImportError: No module named icu). Seeing if there may be a workaround for installing icu on kubernetes, as locally installing in the venv seems to be breaking. A few other people have mentioned that they have also tried installing icu in the venv but were soon confronted with lib issues similar to the ones mentioned earlier.

@Nemo_bis To clarify...

Any lib dependencies issues (like libicui18n's) might as well be rectified with a global installation, so to speak, of icu on both kubernetes and bastion.

To clarify, before the step 1 above, I performed the following steps to install icu in my venv (because icu could not be found came up before doing these steps, i.e. w/o icu installed in the venv, in the ~/uwsgi.log, i.e. after doing webservice --backend=kubernetes python2 start; also, created the venv before doing the following steps via via virtualenv venv in the kubernetes shell within ~/www/python):

1. wget http://download.icu-project.org/files/icu4c/57.1/icu4c-57_1-src.tgz
2. gunzip -d < icu4c-57_1-src.tgz | tar xvf -
3. cd icu/source
4. chmod +x runConfigureICU configure install-sh
5. ./runConfigureICU Linux --prefix=/data/project/spiarticleanalyzer/www/python/venv
6. make
7. make install
8. webservice --backend=kubernetes python2 start
9. Bottom of ~/uwsgi.log reads:

Should https://packages.debian.org/jessie/libs/libicu52 (or another icu lib) be installed on kubernetes (python 2)?

@scfc Ah, okay. Let me clarify a few things.

EDIT: Seems unrelated.

EDIT: Seems unrelated.

Worth looking into. Won't triage any higher until critical errors or widespread effect of this issue is demonstrated.

To note, I'm on @tools-bastion-03.

FYI, extreme lag continues.

Although Wikimedia Maps is not a direct replacement, I'll look into
leafletjs for now.

+ 2017-02-10 23:12 RainbowSprinkles: gerrit: restarting service to https://wikitech.wikimedia.org/wiki/Server_Admin_Log. After the restart, a handful of users have indicated that the service seems to be working for them now.

Quick update, if I may:

Seems to be gerrit?

Just to get an idea of the issue's scope:

Task seems important in that it is affecting users' (such as andrew's) abilities to upload critical patches, such as patches relevant to the issues highlighted by shinken-wm in -labs earlier.

@Andrew @daniel mentioned similar issues over IRC not long ago.

@greg Important enough. Been affecting several users on IRC.

@zhuyifei1999 Okay, so the discussion right now seems to be a) use Wikimedia Maps or b) explain why Wikimedia Maps is not sufficient and request installation of basemap on the grid.

In T157744#3016851, @zhuyifei1999 wrote:

If you build virtualenv on bastion, it's not gonna work on k8s.
If you build virtualenv on k8s interactive, it not gonna work on grid.

Different os versions is not compatible.

The issue of not being able to access basemap while running on the gridengine was brought up. Hence, I've run the following two tests as well:

Pending update from T157744.

Moving to T157744.

Removed previous venv. Rebuilt venv per above. Installed pip install -r requirements.txt. requirements.txt in ~/repo/SPIArticleAnalyzer reads

Another fundamental issue here seems to be that webservice --backend=gridengine uwsgi-python stop yields Your webservice is not running (did leave a substantial amount of time between entering webservice --backend=gridengine uwsgi-python start and webservice --backend=gridengine uwsgi-python stop.

To note, error.log in ~ reads

Also, to note, I tried the following:

@bd808 Entire uwsgi.log file:

@bd808 ~/www/python/src currently symlinks to ~/repo/SPIArticleAnalyzer. I have checked uwsgi.log in ~. See below (from foot of uwsgi.log)

@Dzahn Yes, I am aware of this, but thank you for pointing that out. I performed the command anyway on advice of @scfc. I also already tried building this locally (http://matplotlib.org/basemap/users/installing.html), which didn't seem to resolve the issue (same traceback as the ones presented above).

@scfc Regarding #1 (looking for python-mpltoolkits.basemap), see below:

@scfc Regarding #4 (python version), the python version being used with python is 2.7.6.

@scfc Regarding #3 (installing basemap with pip install basemap), see below:

@Legoktm Perhaps you were referring to using jsub instead of python directly. When I checked python.out and python.err, I don't see any output.

@Legoktm Am I not already on tools-login? @Dzahn I think we seem to be on the same page perhaps.

@Legoktm ssh justberry@tools-login.wmflabs.org still brings me back to justberry@tools-bastion-03:~$. become spiarticleanalyzer yields tools.spiarticleanalyzer@tools-bastion-03:~$. How do I change over to tools-login?

@scfc Has sudo apt-get install python-matplotlib and sudo apt-get install python-mpltoolkits.basemap both been run? I'm not sure if the latter has been run. I get the following when searching for what has already been installed globally:

Just to note, python getAllUsers.py is being run from ~/repo/SPIArticleAnalyzer.

@scfc Ran pip install matplotlib after source $HOME/www/python/virtualenv/bin/activate (in virtualenv). Then, python getAllUsers.py yields

@scfc Previous issue seems to be resolved with a wrapper installation (local vitual environment installation via pip install pyicu). However, the following trackback comes up for python getAllUsers.py.

@scfc icu import issue:

@scfc Thanks, I'll test it out. Also, are you on IRC by any chance? I've been told that your IRC username is scfc, which appears to be offline (not sure if you're under another username atm perhaps).

@scfc Hm...

@bd808 Also, as far as a demo goes, I have performed a demo locally, which works (and successfully generates and saves an image with basemap).

@bd808 Python packages (links now provided), been using kubernetes (so that should work), and you can find the relevant rep (https://github.com/JustBerry/SPIArticleAnalyzer), specifically https://github.com/JustBerry/SPIArticleAnalyzer/blob/master/SPIArticleAnalyzer/getAllUsers.py.

@Yair_rand How might we use Wikidata's diff structure in the API for Wikipedia? It's often easier to list changes on Wikidata because changes are made to specific properties/fields, rather than to bodies of text.

Still ~100 errors. Monitoring...

Email sent to ops-l. Awaiting patch review from ops.

Number of workers needs to be increased to reduce the load.

Investigate channel logs in #wikimedia-operations as necessary: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-operations/20170205.txt

Very important (if not unbreak now).

@scfc Looks like this bug a continuing problem. If that patch works,
perhaps the revision process can begin via gerrit?

I got the same error by doing: