Page MenuHomePhabricator

Moving many (>100) subpages requires loads of memory; leads to PHP Fatal error
Open, LowPublic

Description

$wgMaximumMovedPages has a current default limit of 100. On translatewiki.net we use ini_set( 'memory_limit', '130M' );.

Whenever we try to move a lot of subpages in the MediaWiki namespace (130+) the request fails on "PHP Fatal error: Allowed memory size of 136314880 bytes exhausted (tried to allocate 280258 bytes)" or something similar.

Subpage moving should be made less memory intensive somehow.


Version: 1.15.x
Severity: normal

Details

Reference
bz18585

Event Timeline

bzimport raised the priority of this task from to Low.Nov 21 2014, 10:31 PM
bzimport set Reference to bz18585.
bzimport added a subscriber: Unknown Object (MLST).

Maybe have it break up the move process so that it does a few at a time in batches, that should make it less memory intense, but i don't know how hard that would be.

(In reply to comment #2)

Job queue sounds like a match

I don't think that's a good idea. The limit of 100 subpages was added so that moving a bunch of subpages could reasonably be done in one request, with immediate feedback (success/error). If subpage moves are done through the job queue, there's no way to inform the user when one of the moves fails (e.g. because the target page already exists or is protected).

Could the priority be raised a bit? It seems moving pages with translations is basically broken due to the extremely bad perf. Tested with
MediaWiki 1.35.1
PHP 7.3.19-1~deb10u1 (fpm-fcgi)
MariaDB 10.3.27-MariaDB-0+deb10u1

in our staging instance, trying to move this page with its ~150 subpages: https://wiki.documentfoundation.org/Videos/_Convert_color_code

Word from our admin before he killed the task: "shows no sign of slowing down, 15GiB now and will soon start swapping"

@Buovjaga: No, as priority does not cause reality, it only reflects it... But anyone is very welcome to investigate, obviously.

Aklapper renamed this task from Moving many subpages requires loads of memory to Moving many (>100) subpages requires loads of memory; leads to PHP Fatal error.Dec 19 2020, 10:12 PM
Aklapper removed a subscriber: wikibugs-l-list.

Moving translatable pages is failing with a similar issue, but with a different underlying cause: T260812: Moving of translation page fails with OOM exception (database job queue backend).

We have not seen OOM issues on translatewiki.net for moving non-translatable pages, but it may be because we have switched to using Special:ReplaceText (or Special:ManageMessageGroups of Translate) for mass moves.