bzimport set Reference to bz27727.
Reedy created this task.Via LegacyFeb 25 2011, 11:24 PM
Reedy added a comment.Via ConduitFeb 25 2011, 11:34 PM

Rebuild new version of wikimedia-job-runner and install on job runners, with fix in r82821, r82822

Reedy added a comment.Via ConduitFeb 28 2011, 2:25 PM
  • Bug 27782 has been marked as a duplicate of this bug. ***
Redrose64 added a comment.Via ConduitMar 4 2011, 3:35 PM

The job queue on English Wikipedia has been down for well over a week now and there are over 400,000 jobs stacked up waiting. See http://en.wikipedia.org/wiki/Help_talk:Job_queue#Where.27s_it_gone.3F

If this is not the correct place please advise exactly where we should be filing the bug report.

Reedy added a comment.Via ConduitMar 4 2011, 3:49 PM

They're running, but enwiki being the biggest, will be the last to be dealt with.

So the job runners don't get stuck and stay only on the bigger wikis for a long time, they rank the smaller wikis higher...

It would seem, with like 12k being added to it in a 4 or 5 hour period, there are still big long running tasks going on. Most likely a rename.

A quick look shows that the first enwiki entry is currently a rename...

I'm asking to find out if the job runners are still running fine. Will report back

Rich_Farmbrough added a comment.Via ConduitMar 4 2011, 7:46 PM

This sounds like a recipe for resource starvation. Surely taking age into account is part (I would say it should be all) of the mix?

Bear in mind that from a utilitarian POV a job entry in a big wiki queue is affecting more pages for more people...

If an even simpler sheduler were wanted - count the number of entries in a queue and do n% of them before moving to the next queue. This would get through the backlog at t0 at more or less the same time for each wiki.

Reedy added a comment.Via ConduitMar 4 2011, 8:02 PM

There is no concept of age in the job queue, bar lower job is is older.

I have logged a bug asking for the time stamp to be also logged. So maybe it will be able to in future

Reedy added a comment.Via ConduitMar 4 2011, 8:03 PM

See 27724

Peachey88 added a comment.Via ConduitMar 5 2011, 4:20 AM
  • Bug 27863 has been marked as a duplicate of this bug. ***
bzimport added a comment.Via ConduitMar 6 2011, 12:26 PM

nuuanu wrote:

Do you have a time frame on when the job queue on English Wikipedia will be dealt with? As of March 6, it's getting worse, not better.
http://en.wikipedia.org/wiki/Help_talk:Job_queue#Where.27s_it_gone.3F

Redrose64 added a comment.Via ConduitMar 7 2011, 3:14 PM

Now 529550 (over half a million) pending jobs. We're keeping a periodic update at the en:wp page mentioned in the last post. This works out at about 28000 per day.

demon added a comment.Via ConduitMar 9 2011, 2:13 PM
  • Bug 27953 has been marked as a duplicate of this bug. ***
tstarling added a comment.Via ConduitMar 16 2011, 5:55 AM

The backlog has now cleared and most job runners are sleeping. Marking fixed.

Add Comment