Page MenuHomePhabricator

Consider httpd for quibble instead of php built-in server
Open, NormalPublic

Description

I haven't done extensive tests, but after using httpd and php -S a fair amount for local development, httpd seems to be the winner in performance:

httpd:

❯ ab -n 500 -c 10 http://kafes.local/index.php/Special:BlankPage
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking kafes.local (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:        Apache/2.4.39
Server Hostname:        kafes.local
Server Port:            80

Document Path:          /index.php/Special:BlankPage
Document Length:        50625 bytes

Concurrency Level:      10
Time taken for tests:   45.375 seconds
Complete requests:      500
Failed requests:        342
   (Connect: 0, Receive: 0, Length: 342, Exceptions: 0)
Total transferred:      25449234 bytes
HTML transferred:       25242234 bytes
Requests per second:    11.02 [#/sec] (mean)
Time per request:       907.493 [ms] (mean)
Time per request:       90.749 [ms] (mean, across all concurrent requests)
Transfer rate:          547.72 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:   236  893 188.4    860    1420
Waiting:      205  804 178.2    778    1407
Total:        236  893 188.4    860    1420

Percentage of the requests served within a certain time (ms)
  50%    860
  66%    930
  75%    997
  80%   1054
  90%   1190
  95%   1258
  98%   1324
  99%   1416
 100%   1420 (longest request)

php -S:

❯ ab -n 500 -c 10 http://localhost:8888/index.php/Special:BlankPage
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests


Server Software:
Server Hostname:        localhost
Server Port:            8888

Document Path:          /index.php/Special:BlankPage
Document Length:        51357 bytes

Concurrency Level:      10
Time taken for tests:   203.485 seconds
Complete requests:      500
Failed requests:        499
   (Connect: 0, Receive: 0, Length: 499, Exceptions: 0)
Total transferred:      25502981 bytes
HTML transferred:       25320981 bytes
Requests per second:    2.46 [#/sec] (mean)
Time per request:       4069.707 [ms] (mean)
Time per request:       406.971 [ms] (mean, across all concurrent requests)
Transfer rate:          122.39 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   395 4017 675.9   3881    7155
Waiting:      389 4004 674.6   3872    7150
Total:        395 4018 675.9   3881    7155

Percentage of the requests served within a certain time (ms)
  50%   3881
  66%   4060
  75%   4213
  80%   4369
  90%   4718
  95%   5180
  98%   5968
  99%   6691
 100%   7155 (longest request)

Event Timeline

kostajh created this task.Jun 6 2019, 2:23 PM

Would you be able to run the same benchmark for hhvm as well?
hhvm -m server -p 8888 -d "hhvm.static_file.extensions[svg]=image/svg+xml"

Nice result, I was not expecting php -S to be THAT slower.


side note

On another task, Timo mentioned requests are not compressed, though I don't think it is a performance issue, that shows up that the built-in server uses the PHP options for CLI.

Serving a page through php -S I got:

PHP_SAPIcli-server
opcache.enableOn
opcache.enable_cliOff

But that does not seem to change anything from my loal testing.

end of side note


So yeah, looks like we need Quibble to learn how to start one of Apache/lighttpd/nginx and add that as a dependency.

Would you be able to run the same benchmark for hhvm as well?

I don't have hhvm set up on my machine, but if I get it configured, then sure :)

So yeah, looks like we need Quibble to learn how to start one of Apache/lighttpd/nginx and add that as a dependency.

I started experimenting with this; I started with Apache then abandoned it because of the difficulties of running as non-root, then tried nginx but that has its own issues with kick off php-fpm and nginx from within Quibble.

Would it work to do something like a CLI argument for --webserver=http://apache:8080 in which case Quibble routes browser requests to http://apache:8080. In integration/config we would then need to do docker run for an image containing Apache + PHP-(version), and after Quibble is done we'd need to stop and remove that container.

I started experimenting with this; I started with Apache then abandoned it because of the difficulties of running as non-root, then tried nginx but that has its own issues with kick off php-fpm and nginx from within Quibble.

Apache I could imagine you can not bind to tcp port 80 since that is only for root and I am not sure how that would work in an unprivileged container. That can be worked around with some linux capability, but Quibble / MediaWiki tests already support a port > 1024. So Apache should really be achievable.

Nginx, I have no idea, if you have hints I am willing to learn!

Thinking long, Quibble might well be able to detect/pick a httpd server depending on whats is available in the environment (eg Apache > nginx > lighttpd > php -s).

Would it work to do something like a CLI argument for --webserver=http://apache:8080 in which case Quibble routes browser requests to http://apache:8080. In integration/config we would then need to do docker run for an image containing Apache + PHP-(version), and after Quibble is done we'd need to stop and remove that container.

Then we will have to orchestrate containers and I am not really willing to do that with the current CI stack. The same issue I had with the patch to spawn Parsoid as a backend service :-\ Really that should be implemented with Helm and Kubernetes.

I am pretty sure we can hook Apache from Quibble :]

Change 516729 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[integration/quibble@master] (wip) Add option for using Apache as server

https://gerrit.wikimedia.org/r/516729

Preliminary test run with Apache was not that thrilling: the selenium page.js test passed in 300 seconds instead of 345 with PHP built in server. I’ll poke at it some more though.

Some benchmarks with quibble + this patch + docker on macOS.

Starting the container

cd ~/quibble-builds && rm src/LocalSettings.php; docker run -it --rm \
  --name quibble \
  --entrypoint=bash \
    -e ZUUL_PROJECT=mediawiki/core \
    -v /Users/kharlan/quibble-builds/cache:/cache \
    -v /Users/kharlan/quibble-builds/log:/workspace/log \
    -v /Users/kharlan/quibble-builds/ref:/srv/git:ro \
    -v /Users/kharlan/quibble-builds/src:/workspace/src \
    quibble-apache

built-in PHP + sqlite

quibble --skip-deps --skip-zuul --db sqlite --db-dir /workspace/src/cache/ --run selenium

run 1

15 passing (318.30s)
5 skipped

run 2

15 passing (299.40s)
5 skipped

run 3 (disabled page undoable test):

14 passing (302.40s)
6 skipped

apache2 + sqlite

quibble --skip-deps --skip-zuul --db sqlite --db-dir /workspace/src/cache/ --run selenium --webserver=apache2

run 1

14 passing (246.50s)
5 skipped
1 failing

run 2

14 passing (208.10s)
5 skipped
1 failing

run 3 (disabled page undoable test)

14 passing (263.40s)
6 skipped

This error occurred on both runs with --webserver=apache2, so I disabled it for the third test run:

1) Page should be undoable:
'beforeEach-content-0.8998667415652057-Iñtërnâtiônàlizætiøn' === 'beforeEach-content-0.0023315024241947757-Iñtërnâtiônàlizætiøn'

built-in PHP + MySQL

quibble --skip-deps --skip-zuul --db mysql --db-dir /workspace/src/cache/ --run selenium

run 1

14 passing (225.20s)
6 skipped

run 2

14 passing (264.80s)
6 skipped

apache2 + MySQL

n.b. For MySQL, I had to change the db directory to /workspace/src/cache.

quibble --skip-deps --skip-zuul --db mysql --db-dir /workspace/src/cache/ --run selenium --webserver=apache2

run 1

14 passing (191.90s)
6 skipped

run 2

14 passing (195.10s)
6 skipped

Change 516729 abandoned by Kosta Harlan:
Add option for using Apache as server

Reason:
IMO let's wait for local-charts :) Although if someone wants to pick this up, feel free

https://gerrit.wikimedia.org/r/516729

Change 516729 restored by Kosta Harlan:
Add option for using Apache as server

https://gerrit.wikimedia.org/r/516729