Page MenuHomePhabricator

refreshlinks.php slow in CVS 1.4
Closed, ResolvedPublic

Description

Author: ameschede

Description:
refreshLinks.php from MediaWiki 1.3.x used to process about 800 pages per 30
seconds on my server, but with 1.4 CVS (beta 1) it comes down to an unacceptable
rate of not even 50 pages per minute.


Version: 1.4.x
Severity: normal
OS: Linux
Platform: PC

Details

Reference
bz991

Related Objects

StatusSubtypeAssignedTask
ResolvedNone
ResolvedNone

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 21 2014, 7:06 PM
bzimport set Reference to bz991.
bzimport added a subscriber: Unknown Object (MLST).

Might be a parser cache-clearing oddity. Try setting in LocalSettings.php:
$wgEnableParserCache = false;

If that doesn't affect it, I'll do some profiling runs when I have a chance.

ameschede wrote:

(In reply to comment #1)

Might be a parser cache-clearing oddity. Try setting in LocalSettings.php:
$wgEnableParserCache = false;

Setting $wgEnableParserCache = false didn't improve the performance.

al2.baeckeroot wrote:

On version 1.3.8 it is very slow also

  • 30 mins for 5000 articles on Athlon 2Ghz + 750 Mo
  • and a poor HD 20MB/s (real mesurement)
  • and no log for mysql (40% longer with huge logs for refreshLinks ;)

I checked many things, but know very little ;)

cur is InnoDB here.
It seems to have huge impact on some performance... i don't know !

The cache seems to be inefficient, eats up memory but doenst provide
speed improvement: when i split the job, the total time is the same!

I just did a refreshLinks.php which clocked 1000 pages per minute on a 2GHz
Athlon with 512MB memory, on a database with ~32200 pages (nostalgiawiki).

This is an order of magnitude over the reported slow speeds and within an order
of magnitude of the reported fast speeds (considering variation from CPU,
configuration, disk speeds, size of articles, etc a more direct comparison of the
numbers is not possible offhand).

Previous extreme slowness was likely caused by the totally broken bits causing
bug 1132, which would have hugely inflated the number of links being
manipulated.

If you can still see unreasonable slowness after updating to get the fix for 1132,
please reopen with additional data. Resolving as fixed...