In T222985#7163999, @ArielGlenn wrote:

lbzip2 decompresses in parallel as well. We use that for compression of the SQL/XML dumps.

I've moved the dump files to scratch for now, hope this helps. I also manually cleaned up some big dumps.

Sorry for that, I'll look into automating the cleanup.

Your legacy URL ingress is also not including the same rewrite rules as webservice would create.

Are these rewrite rules documented somewhere? This was one of the troubles I had getting started with using webservice directly, in that I have no idea how ingress works and what is rewritten (is the /$TOOL name removed from the URL before passing the request to my app or is it not?). The only reference I could find to this was https://wikitech.wikimedia.org/wiki/Help:Toolforge/Web#Other_/_generic_web_servers, but that is at the end of a very long article and also mixes old grid engine and kubernetes parts.

Thanks, I cleaned up the k8s configs and it works fine again now. I might consider switching to the webservice tooling later, however I also like using raw k8s since it is easier to understand what is going on and modify things as necessary.

The tool never used the Webservice command launcher. It starts its own uwsgi process using k8s. I will look into what's wrong. It definitely used to work.

This does not seem fully fixed yet: https://www.wikidata.org/wiki/Wikidata_talk:SPARQL_query_service#Possible_bug. Example from that post:

This query https://query.wikidata.org/#SELECT%20%3Fprop%20%3Ftype%20WHERE%20%7B%20%3Fprop%20wikibase%3ApropertyType%20%3Ftype%20FILTER%20%28CONTAINS%28STR%28%3Fprop%29%2C%22Q%22%29%20%26%26%203%21%3D1%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%7D still returns wrong results. Has WDQS not updated yet?

But I can do a zstd decompression -> zstd compression test.

I don't have enough disk space for a compression test, that's correct.

Now the same with zstd:

So I tried lbzip2, here's the result (on a VM sever with 2 cores, 2.1GHz, the decompression is CPU bound):