Articles.py is using a lot of IO. Apparently it is reading its sqlite db over and over in a quick loop.
Yesterday it caused High iowait in tools-exec-1419 this morning it was run from tools-bastion-03 and caused high io use there as well, making it slow.
Pherhaps this could be migrated to use toolforge users databases over at tools.db.svc.eqiad.wmflabs?
Hi Chicocvenancio ,
Thanks for reporting.
It is normally running on the grid each night. This morning, I ran it on bastion-03 exceptionally, in order to debug a feature, but you killed it (and I perfectly understand why, so I will debug it locally).
Maybe I can begin with looking at my code to see how I can decrease the IO (I can probably add a cache in memory to decrease the queries made to the SQLite DB).
Best regards
The only file system that your job will consistently have access to is an NFS share. Using an SQLite database over NFS is going to continue to cause network IO problems for both the client and server. Please use a user database on tools.db.svc.eqiad.wmflabs instead.
Assuming this is about https://tools.wmflabs.org/powow/ (there is no URL in the task description).
@Peter17: Hi! This task has been assigned to you a while ago. Could you maybe share an update? Do you still plan to work on this task? Thanks in advance!
@Aklapper I removed most of my project from the cluster several months ago and I host it on my own server ; I just host simple tasks on wmflabs for this project. Regards