Currently, WDQS has a testing setup on http://wdqs-test.wmflabs.org. However, we are experiencing both storage and performance issues there. Max storage on labs right now is 160G, and WDQS database size is rapidly outgrowing this limitation.
Currently, the growth rate for the database is about 75% a year. Production database is now 193G. That means even if we double the labs instance storage, it would only be enough for another year, or maybe two if we store reduced data set. So, we need to find a solution for a test setup.
Ways for solving it:
- Create a huge storage labs instance (300G, 500G) template
- Store reduced data set for testing, e.g. only truthy data, only English labels, no sitelinks, etc.
- Migrate test setup to real hardware outside of labs (maybe less high-powered than production hosts) - maybe to labs-support like relforge
This task is to explore and evaluate possible solutions for this issue.