Page MenuHomePhabricator

Job queue not properly worked when using cron job and visiting pages before execution
Closed, DeclinedPublic

Description

Following the documentation at https://www.mediawiki.org/wiki/Manual:Job_queue, I have changed my configuration to set $wgJobRunRate to 0, and to execute maintenance/runJobs.php every 20 minutes. Ever since I did that, updates in a template are not reliably propagated to the pages that use them any more.
I can tell from the mails I got that the cron job is executed. I can sometimes reproduce the issue by

  • Editing a template
  • Running showJobs.php to confirm there are jobs around
  • Looking at a page that uses the template to confirm the update has not been propagated yet
  • Running runJobs.php manually. This prints
2015-10-31 11:45:42 htmlCacheUpdate Vorlage:Projekt pages=array(3) rootJobSignature=36cb29df2e2c1efa7f71f81ced6d82d6cb787caa rootJobTimestamp=20151031114524 STARTING
2015-10-31 11:45:42 htmlCacheUpdate Vorlage:Projekt pages=array(3) rootJobSignature=36cb29df2e2c1efa7f71f81ced6d82d6cb787caa rootJobTimestamp=20151031114524 t=2 good
2015-10-31 11:45:42 refreshLinks Vorlage:Projekt table=templatelinks recursive=1 rootJobSignature=5fcf15e698c59c866e364e221d107418ae00e9af rootJobTimestamp=20151031114524 STARTING
2015-10-31 11:45:42 refreshLinks Vorlage:Projekt table=templatelinks recursive=1 rootJobSignature=5fcf15e698c59c866e364e221d107418ae00e9af rootJobTimestamp=20151031114524 t=3 good
2015-10-31 11:45:42 refreshLinks Vorlage:Projekt pages=array(1) rootJobSignature=5fcf15e698c59c866e364e221d107418ae00e9af rootJobTimestamp=20151031114524 masterPos= STARTING
2015-10-31 11:45:42 refreshLinks Vorlage:Projekt pages=array(1) rootJobSignature=5fcf15e698c59c866e364e221d107418ae00e9af rootJobTimestamp=20151031114524 masterPos= t=163 good
2015-10-31 11:45:42 refreshLinks Vorlage:Projekt pages=array(1) rootJobSignature=5fcf15e698c59c866e364e221d107418ae00e9af rootJobTimestamp=20151031114524 masterPos= STARTING
2015-10-31 11:45:42 refreshLinks Vorlage:Projekt pages=array(1) rootJobSignature=5fcf15e698c59c866e364e221d107418ae00e9af rootJobTimestamp=20151031114524 masterPos= t=82 good
2015-10-31 11:45:42 refreshLinks Vorlage:Projekt pages=array(1) rootJobSignature=5fcf15e698c59c866e364e221d107418ae00e9af rootJobTimestamp=20151031114524 masterPos= STARTING
2015-10-31 11:45:42 refreshLinks Vorlage:Projekt pages=array(1) rootJobSignature=5fcf15e698c59c866e364e221d107418ae00e9af rootJobTimestamp=20151031114524 masterPos= t=94 good
  • Going to the page and confirm that the update *still* has not been propagated properly.

If, however, I do *not* visit the page that uses the template nor check showJobs.php between editing the template and executing the job, then everything works out. Or maybe that's just a coincidence, because unfortunately, this issue is not reproducible. It just silently and unreliably breaks the wiki.
During my experiments, I also sometimes saw updates being propagated without runJobs.php being run. Is that normal, or does this mean that something is acting weird somewhere?

This is using MediaWiki 1.23.11. For now, I'm going to add another cronjob that touches LocalSettings.php every night, just to make sure the wiki is in a consistent state at least once a day.

Event Timeline

Ralfjung-mediawiki raised the priority of this task from to Needs Triage.
Ralfjung-mediawiki updated the task description. (Show Details)
Ralfjung-mediawiki subscribed.

I can't reproduce following those steps.

Maybe it's a caching problem? Content may be cached on your browser, intermediate proxy, etc. Can you describe what's your setup regarding of cache (cache settings in LocalSettings.php, intermediate proxies, etc)?

There are no intermediate proxies. I have been hitting F5 in the browser for all tests. I have no cache configured in LocalSettings - all I found is this commented-out line:

#$wgCacheDirectory = "$IP/cache";

Should I try again using Ctrl+F5 in the browser?

Oh my... I've reproduced the issue.

Following those steps as non-logged in user, the page gets the changes on reload.

But if do the same while being logged-in (from viewing the page the first time, until refreshing after editing the template and running jobs), the page doesn't get the changes on the template even with hard reload on the browser (CTRL+SHIFT+R). Inspecting network traffic, what I see when reloading the page is that the server responds with a HTTP 304 - Not Modified

Note that $wgDebugToolbar must not be true, otherwise changes are always reflected on the page.

This happens on 1.23 but doesn't happen on 1.25

Phew, so I'm not hallucinating here :)

Indeed I was logged in all the time. Editing pages is not allowed for anonymous users in the wiki I'm running. Sorry I forgot to mention this.

Krinkle subscribed.

Job queue running was improved a lot in 1.28. This was also backported to 1.27 and 1.25, but not 1.23. Declining since such backport is non-trivial and 1.23 LTS is only supported for one more month per https://www.mediawiki.org/wiki/Version_lifecycle.

Please upgrade to at least the 1.27 LTS.