Update 2018/01/02: Findings
We now know that we're looking at ~1 month to get the headless Chromium based render service deployed. It might take longer to get it undeployed if it isn't a viable replacement for the Electron-based render service. If we have an understanding of how the new service performs ahead of time, then we might be able to save folk (including us) a lot of time.
From the parent task:
Performance testing on a VPS isn't ideal but it's cheaper than testing on production hardware (the time cost of deploying the service is ~1 month). @bmansurov has already set up a VPS to test the service, which is accessible here: http://chromium-pdf.wmflabs.org.
List of test articles
https://en.wikipedia.org/wiki/Battle_of_Mosul_(2016%E2%80%932017) - long article, lots of images
https://en.wikipedia.org/wiki/Hendrick_Motorsports - long article, tables, images
https://en.wikipedia.org/wiki/Panama_Papers - long article, images
https://en.wikipedia.org/wiki/List_of_compositions_by_Franz_Schubert - long list article
top 5 printed articles
long right to left articles
Things to double check
During T178501#3767355 @pmiazga found out that some rendered PDFs are incomplete. This happens only when the queue is full and service is constantly handling at least 2-3 concurrent requests at once. During performance testing please verify that all PDF files are rendered correctly and contain all pages.
- Generate PDFs using the list of articles above. Verify that the PDF contents look good, i.e. the PDFs contain actual article text, and not some error message from RESTBase or somewhere else. Upload the resulting PDFs here.
See T178278#3854820 onwards.
- Measure and report times spent rendering articles in succession. The service logs contain this information.
- Use siege to report the performance of the service. Create various scenarios where you have a combination of articles (from above) and control the number of concurrent requests. Analyze the service logs to make sure that nothing unexpected happens. For example, if the concurrency in siege is set as 5 and it's set in service as 3, then make sure that after the 3rd request, the next two are aborted immediately.
- Monitory the system load while doing these tests. See T175853#3616304 for reference.