There were issues with the wiki getting slow and/or unresponsive when multiple users were logged in simultaneously. This seem to happen with at little as three users. The server load doesn't seem to be an issue, with CPU between 5 and 25 % (with some higher spikes) and RAM at 50%.
Description
Related Objects
- Mentioned In
- T191385: Sub contract Kreablo AB for Samsyn
- Mentioned Here
- T191385: Sub contract Kreablo AB for Samsyn
Event Timeline
Logfile is now saved to C:\Logs\debug-samsyn.log, as per https://www.mediawiki.org/wiki/Manual:How_to_debug#Logging.
I have asked for help with this in mediawiki-l: https://lists.wikimedia.org/pipermail/mediawiki-l/2018-March/047270.html.
I noticed that you get more info about what's actually going on in case of HTTP error, if you try to access the wiki from inside the server. In this instance it turned out that a debugging extension for PHP was the culprit, but this should be investigated next time the issue occurs.
Also: PHP error log has been turned on and is viewable in Event Viewer.
Two pieces of possibly relevant information from the Event Viewer:
- System - Provider [ Name] MySQL - EventID 100 [ Qualifiers] 49152 Level 4 Task 0 Keywords 0x80000000000000 - TimeCreated [ SystemTime] 2018-03-12T13:17:02.538367400Z EventRecordID 204601 Channel Application Computer vpsamv-p-srv01.samverkan.su.se Security - EventData Aborted connection 4902 to db: 'samsyn' user: 'root' host: 'localhost' (Got an error reading communication packets)
The connection number varies.
And:
- System - Provider [ Name] PHP-7.2.1 - EventID 2 [ Qualifiers] 16384 Level 4 Task 6 Keywords 0x80000000000000 - TimeCreated [ SystemTime] 2018-03-12T13:19:59.127987600Z EventRecordID 204622 Channel Application Computer vpsamv-p-srv01.samverkan.su.se Security - EventData php[7476] PHP Fatal error: Uncaught MWException: Web server doesn't provide either REQUEST_URI, HTTP_X_ORIGINAL_URL or SCRIPT_NAME. Report details of your web server configuration to https://phabricator.wikimedia.org/ in C:\inetpub\wwwroot\mediawiki\includes\WebRequest.php:798 Stack trace: #0 C:\inetpub\wwwroot\mediawiki\includes\WebRequest.php(828): WebRequest::getGlobalRequestURL() #1 C:\inetpub\wwwroot\mediawiki\includes\exception\MWExceptionHandler.php(423): WebRequest->getRequestURL() #2 C:\inetpub\wwwroot\mediawiki\includes\exception\MWExceptionHandler.php(516): MWExceptionHandler::getURL() #3 C:\inetpub\wwwroot\mediawiki\includes\exception\MWExceptionHandler.php(597): MWExceptionHandler::getStructuredExceptionData(Object(ErrorException), 'mwe_handler') #4 C:\inetpub\wwwroot\mediawiki\includes\exception\MWExceptionHandler.php(657): MWExceptionHandler::jsonSerializeException(Object(ErrorException), false, 3, 'mwe_handler') #5 C:\inetpub\wwwroot\mediawiki\includes\exception\MWExceptionHandler.php(195): MWExceptionHandler::logError( in C:\inetpub\wwwroot\mediawiki\includes\WebRequest.php on line 798
The trace can look a bit different, but the final call is always the same.
Contrary to previous observations, the CPU seem to be running at a quite high percentage when multiple requests are handled simultaneously. I don't think I had been looking at it right when the requests are received before.
Just loading recent changes three times in parallel causes the CPU usage to shoot to 100% for a while, which seems odd. This is with all extensions disabled. I'll do some investigation into whether we've missed something with caching or anything else that can impact the performance, but it may be worth adding a couple more CPU:s.
@SaraMortsell, could you check with SU if it's reasonable to upgrade the server with two more CPU cores (to a total of four)? If this doesn't help (or only allows a few more users at once), it's an indication that something else is the problem.
Two more CPU did not make any difference.
Another suggestion: https://forums.iis.net/t/1200661.aspx?Slow+load+of+Mediawiki+landing+page+in+IIS8+using+PHP+MySQL+FastCGI
Är det något knas när Mediawiki kommunicerar med databasen? Den här sidan föreslår att man pekar på IP-adressen 127.0.01 i stället för localhost för att det ska gå snabbare.
I followed the instructions at https://forums.iis.net/t/1200661.aspx?Slow+load+of+Mediawiki+landing+page+in+IIS8+using+PHP+MySQL+FastCGI and https://www.charlesrcook.com/archive/2012/09/12/mediawiki-iis7-output-caching-and-friendlyshort-urls.aspx, but there was no improvement.
These issues will be solved by moving the Wiki to a Linux server and sub contracting Kreablo to do it T191385: Sub contract Kreablo AB for Samsyn