The new media-list endpoint is a variation on the existing media endpoint that contains only media info derived from the page HTML. It is intended to avoid the performance and invalidation problems posed by the existing media endpoint.
Proposed path: /page/media-list/{title}{/revision}
I'll add some local benchmarking numbers below.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | • Mholloway | T225443 Media endpoint does not refresh structured captions | |||
| Resolved | • mobrovac | T226105 Deploy new media-list endpoint in RESTBase |
Local ab numbers when fetching page HTML over HTTP from the public RESTBase endpoint:
mholloway@mholloway:~/code/wikimedia/mobileapps$ ab -c 1 -n 1000 http://localhost:6927/en.wikipedia.org/v1/page/media-list/San_Francisco
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software:
Server Hostname: localhost
Server Port: 6927
Document Path: /en.wikipedia.org/v1/page/media-list/San_Francisco
Document Length: 18015 bytes
Concurrency Level: 1
Time taken for tests: 384.439 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 18934000 bytes
HTML transferred: 18015000 bytes
Requests per second: 2.60 [#/sec] (mean)
Time per request: 384.439 [ms] (mean)
Time per request: 384.439 [ms] (mean, across all concurrent requests)
Transfer rate: 48.10 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 328 384 27.3 390 522
Waiting: 328 384 27.3 390 522
Total: 328 384 27.3 390 522
Percentage of the requests served within a certain time (ms)
50% 390
66% 401
75% 404
80% 406
90% 412
95% 424
98% 448
99% 466
100% 522 (longest request)And when grabbing stored page HTML from a local RESTBase:
mholloway@mholloway:~/code/wikimedia/restbase$ ab -c 1 -n 1000 http://localhost:6927/en.wikipedia.org/v1/page/media-list/San_Francisco
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software:
Server Hostname: localhost
Server Port: 6927
Document Path: /en.wikipedia.org/v1/page/media-list/San_Francisco
Document Length: 18015 bytes
Concurrency Level: 1
Time taken for tests: 245.354 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 18925000 bytes
HTML transferred: 18015000 bytes
Requests per second: 4.08 [#/sec] (mean)
Time per request: 245.354 [ms] (mean)
Time per request: 245.354 [ms] (mean, across all concurrent requests)
Transfer rate: 75.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 181 245 41.1 244 844
Waiting: 181 245 41.1 244 844
Total: 181 245 41.1 244 845
Percentage of the requests served within a certain time (ms)
50% 244
66% 259
75% 267
80% 271
90% 286
95% 309
98% 340
99% 371
100% 845 (longest request)Mentioned in SAL (#wikimedia-operations) [2019-06-24T21:04:07Z] <mobrovac@deploy1001> Started deploy [restbase/deploy@a915f69]: Add /page/media-lint - T226105 - and various other cleanups
Mentioned in SAL (#wikimedia-operations) [2019-06-24T21:23:14Z] <mobrovac@deploy1001> Finished deploy [restbase/deploy@a915f69]: Add /page/media-lint - T226105 - and various other cleanups (duration: 19m 08s)