MediaWiki does a lot of work to try and do this by default, yet it seems the particular combination of factors in CI make it not enable gzip for web responses.
This means that behaviour is not very representative. In particular, it means that for Fresnel, the metrics "raw code" (decodedBodySize) and "network transfer" (encodedBodySize) are the same, whereas the latter is meant to be smaller and behave differently.
I thought maybe that our use of the PHP built-in webserver (php -S) is causing this, our the maintenance/dev/router.php script that we use.
However, I'm not able to reproduce uncompressed responses locally with these. They are compressed right out of the box. MediaWiki detects that zlib is installed, and detects that no default compression is happening elsewhere, and then rolls its own (handled in OutputHandler.php, called from WebStart.php).
This rules them out as the cause of the problem.
Using a Quibble container locally, I was able to reproduce the issue. Starting php -S from a new non-mediawiki directory in the container, I find that there is a pre-existing output buffer. Hence, MediaWiki would be unable to start its own top-level output buffer.
The problem is caused by the following IN setting for PHP:
; Default Value: Off ; http://php.net/output-buffering output_buffering = 4096
# ob_list_handlers() array( 0 => "default output handler" )
Looks like the output_buffering feature is enabled. This means PHP starts an output buffer natively that flushes output directly to the user in chunks of 4KB.
By default in PHP this is disabled. I've also checked WMF production and it is disabled there as well, both for HHVM and for PHP 7.2.
When disabling it, the array from ob_list_handlers() becomes empty, and MediaWiki's output compression starts working.
Quibble indeed uses php -S which saved me to have to deal with a frontend web server and have to configure it. Anyway, since the webserver is the PHP CLI it eventually loads /etc/php/7.0/cli/php.ini
Should we have Quibble to start the server hardcoding the setting? Eg:
php -d output_buffering=Off -S
Which might be recognized by HHVM as well (to be checked).
@hashar Maybe, yeah!
I am also wondering where that INI file is coming from, because afaik it is not the PHP default. Perhaps that could be disabled somehow. But, I think it's nice to set explicitly on the php call, that way it will not depend on the environment.
The php package should be https://packages.debian.org/source/stretch/php7.0 , with the git repository supposedly on gitlab:
https://gitlab.com/deb.sury.org/php
https://gitlab.com/deb.sury.org/php.git
Which yield: You need to sign in or sign up before continuing.
I guess it is a case in favor of hosting on a third party.
Anyway I have instead downloaded the source package (apt-get source php7.0). To the best I can tell, php-7.0.33 upstream tarball comes with php.ini that have output_buffering = 4096.
So yeah, should be hardcoded in Quibble instead :]
(Still interested in this as deliverable for a future quarter - as part of Fresnel maintenance and on-going improvements. But, if anyone else is interested in this before I get to it, that'd be great too.)
Change 534468 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[integration/quibble@master] Disable output buffering in running php
Change 535938 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/core@master] DevelopmentSettings: Disable output buffering to be more like production
Ok I gave it a try creating a phpinfo file at the root of the mediawiki/core checkout and running something like:
rm src/LocalSettings.php quibble --skip-zuul --skip-deps --db sqlite -c 'xdg-open http://127.0.0.1:9412/phpinfo.php'
output_buffering shows with no value.
And a basic script shows that it is no more buffered:
<?php $i = 0; while ( true ) { print $i++ . "\n"; sleep(1); if ( $i === 1000 ) { exit; } }
Change 534468 merged by jenkins-bot:
[integration/quibble@master] Disable output buffering in running php
Mentioned in SAL (#wikimedia-releng) [2019-09-17T15:32:07Z] <hashar> Tagging quibble 0.0.35 48f6b1d8db9eed8572f255fe86f920753149153c # T225730 T231862 T219694
Change 537469 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] docker: rebuild for quibble 0.0.35
Change 537469 merged by jenkins-bot:
[integration/config@master] docker: rebuild for quibble 0.0.35
Change 537485 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] jjb: Upgrade all quibble jobs to 0.0.35
Change 537485 merged by jenkins-bot:
[integration/config@master] jjb: Upgrade all quibble jobs to 0.0.35
Confirmed to work.
Fresnel run from last night:
| Total size of transfers during page load (pageWeight) | 1.3 MB (± <0.4 B) | 1.3 MB (± <0.4 B) | | Transfer size of HTML document (html) | 30.7 kB (± <0.4 B) | 30.7 kB (± <0.4 B) | | Transfer size of CSS resources (css) | 51.6 kB (± 0 B) | 51.6 kB (± 0 B) | | Transfer size of JavaScript resources (js) | 1.2 MB (± 0 B) | 1.2 MB (± 0 B) | | Transfer size of Image document (img) | 28.1 kB (± 0 B) | 28.1 kB (± 0 B) |
Fresnel run from just now
| | Total size of transfers during page load (pageWeight) | 271 kB (± <0.8 B) | 269 kB (± <0.7 B) | | Transfer size of HTML document (html) | 7.4 kB (± <0.8 B) | 7.4 kB (± <0.7 B) | | Transfer size of CSS resources (css) | 11.6 kB (± 0 B) | 11.6 kB (± 0 B) | | Transfer size of JavaScript resources (js) | 224 kB (± 0 B) | 222 kB (± 0 B) | | Transfer size of Image document (img) | 28.1 kB (± 0 B) | 28.1 kB (± 0 B) |
Clearly much smaller, which can only be explained by gzip.
Well done, and thank you for the verification.
Released and deployed on September 17th with Quibble 0.0.35.
Also confirmed from mw-debug-www.log:
wfClientAcceptsGzip: client accepts gzip. MediaWiki\OutputHandler::handleGzip() is compressing output
Change 535938 abandoned by Kosta Harlan:
DevelopmentSettings: Disable output buffering to be more like production
Reason:
Boldly abandoning to clean up the patch queue a bit. Please reopen if you disagree!