Page MenuHomePhabricator
Create Task
Maniphest T293828

Not Wikilambda test wiki exceeding 1G of memory on main page
Closed, InvalidPublic
Actions

Description

The Not Wikilambda test wiki currently can’t load the main page – I bumped the maximum PHP memory to 1 GiB(!) and it’s still running out:

error.log
2021-10-19 20:40:01: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 135168 bytes) in /data/project/notwikilambda/public_html/w/extensions/WikiLambda/includes/Registry/ZTypeRegistry.php on line 291
2021-10-19 20:40:44: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 20480 bytes) in /data/project/notwikilambda/public_html/w/vendor/symfony/yaml/Parser.php on line 1102

I don’t remember what the memory limits in production are (and if this would already exceed them), but I think this deserves some looking into.

Note that some other pages work fine:

Related Objects
Search...

Event Timeline

LucasWerkmeister created this task.Oct 19 2021, 8:48 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 19 2021, 8:48 PM
LucasWerkmeister added a comment.Oct 19 2021, 8:55 PM

I removed the transclusion of Special:PrefixIndex/Z via the API sandbox and that makes the main page render relatively quickly again. Consistently with this, directly visiting Special:PrefixIndex/Z also runs out of memory.

Stashbot added a comment.Oct 19 2021, 8:56 PM

Mentioned in SAL (#wikimedia-cloud) [2021-10-19T20:56:41Z] <wm-bot> <lucaswerkmeister> previous log messages all related to T293828

LucasWerkmeister added a comment.Oct 19 2021, 8:58 PM

IMHO at this point that’s something for the Abstract Wikipedia team to investigate: why listing a bunch of ZObject pages requires so much memory. (Let me know if you need more details about the wiki setup.)

LucasWerkmeister added a comment.EditedOct 19 2021, 9:04 PM

Along similar lines, loading recent changes with larger limits (e.g. 50 changes, 7 days) is also broken, though stricter limits (e.g. 1 change or 1 hour) work fine. I’m not sure how many of these error lines belong to RecentChanges requests (the first one is a main page request, 20:55 matches the previous Phabricator comment), but at least some of the RecentChanges requests resulted in timeout instead of OOM:

error.log
2021-10-19 20:55:21: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 20480 bytes) in /data/project/notwikilambda/public_html/w/vendor/wikimedia/php-session-serializer/src/Wikimedia/PhpSessionSerializer.php on line 380
2021-10-19 20:58:45: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 20480 bytes) in /data/project/notwikilambda/public_html/w/extensions/WikiLambda/includes/Registry/ZTypeRegistry.php on line 291
2021-10-19 21:00:26: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /data/project/notwikilambda/public_html/w/vendor/opis/json-schema/src/ValidationResult.php on line 133
2021-10-19 21:00:46: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /data/project/notwikilambda/public_html/w/vendor/symfony/yaml/Parser.php on line 655
2021-10-19 21:02:55: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /data/project/notwikilambda/public_html/w/vendor/symfony/yaml/Parser.php on line 1102
2021-10-19 21:03:06: (mod_fastcgi.c.421) FastCGI-stderr: PHP Fatal error:  Maximum execution time of 30 seconds exceeded in /data/project/notwikilambda/public_html/w/extensions/WikiLambda/includes/ZObjectContent.php on line 242
2021-10-19 21:03:20: (http-header-glue.c.1250) read(): Connection reset by peer 7 20
2021-10-19 21:03:20: (gw_backend.c.2149) response not received, request sent: 1822 on socket: unix:/var/run/lighttpd/php.socket.notwikilambda-1 for /w/index.php?title=Special:RecentChanges&hidebots=1&limit=50&days=7&enhanced=1&urlversion=2, closing connection

IMHO that falls under the same category of “listing enough ZObject pages is more expensive than it should be”.

LucasWerkmeister added a project: Abstract Wikipedia team.Oct 19 2021, 9:09 PM

(adding general tag, phase to be determined)

DVrandecic moved this task from To Triage to Phase θ – Throttling on the Abstract Wikipedia team board.Oct 20 2021, 4:20 PM
DVrandecic edited projects, added Abstract Wikipedia team (Phase θ – Throttling); removed Abstract Wikipedia team.
Jdforrester-WMF added a parent task: T303353: Adjust system design to avoid significant performance bottlenecks.Mar 8 2022, 11:11 PM
Jdforrester-WMF added a project: Performance Issue.May 24 2022, 6:53 PM
Jdforrester-WMF moved this task from Incoming to Ready to do on the Abstract Wikipedia team (Phase θ – Throttling) board.
ori closed this task as Invalid.Aug 4 2022, 2:49 PM
ori subscribed.

I profiled a request to Special:PrefixIndex/Z on the Beta Cluster. The profile shows peak memory usage of 18MB.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits