We now know that we're looking at ~1 month to get the headless Chromium based render service deployed. It might take longer to get it undeployed if it isn't a viable replacement for the Electron-based render service. If we have an understanding of how the new service performs ahead of time, then we might be able to save folk (including us) a lot of time.
# Open Questions
1. What?
From the parent task:
>>! In T176627#3680183, @phuedx wrote:
> We should be providing median and 95/99th percentile timings for the service rendering S/M/L/XL/XL… pages and resource consumption on the server during those test runs. Moreover, these tests should be repeatable (i.e. there's a script that we can run by anyone [and at different times!]).
>
> We all acknowledge that we'd have to take the results [[ https://en.wikipedia.org/wiki/Grain_of_salt | with a grain of salt ]] as we'd be using a VPS-hosted instance but they'll be helpful in the interim while we're trying to get the service into production.
>>! In T176627#3684846, @GWicke wrote:
> It might be worth focusing more on robustness than simple-page latency, as that is the more critical issue with Electron. Previously, I tested with a few very large articles (see T142226#2537844). This tested timeout enforcement. Testing with a simulated overload (many concurrent requests for huge pages) could also be useful to ensure that concurrency limits and resource usage limits are thoroughly enforced.
2. Where?
Performance testing on a VPS isn't ideal but it's cheaper than testing on production hardware (the time cost of deploying the service is ~1 month). @bmansurov has already set up a VPS to test the service, which is accessible here: http://chromium-pdf.wmflabs.org.
Per T178278#3726445, we also have access to bare metal with a big pipe. @pmiazga would be required to set up the service on that server.