During a switchover, we want to clean up the caches in the inactive datacenter to avoid for stale data. As a consequence, we need to warm up the caches in the inactive dc after the wipe, to avoid various inconveniences.
Last time we did a switchover it was done using apache-fast-test and a list of URLs that @Krinkle prepared. The issue is that apache-fast-test can perform the requests in parallel on each server (which is needed for populating APC), but only does that sequentially. If those urls take significant time to render (and they do) the total time taken for the warmup can be quite long, in the order of 15 minutes. Since every HHVM server can sustain much more than one request in parallel, we'd need a warmup tool that does the following:
- perform the minimum possible number of requests to warm up the caches satisfactorily
- possibly different list of URLs for the API servers and normal appservers, imagescalers, etc.
- be able to run the list of requests in parallel for each server
- be able to run the list of requests with a configurable concurrency on each server
We can then experiment with this tool to find the concurrency that allows us to warm up the caches in the minimum amount of time.