We generated two sample datasets of Blazegraph logged queries, rewritten to WDQS v2 compatible SPARQL.
As a starting point, we have 3_448_194 rewrites from the internal endpoint, and 2_244_361 rewrites for the external endpoint.
We can use these datasets to replay traffic deterministically, and collect request and system metrics on the WDQS v2 pilot endpoint. In particular, we can compare latency (distributions) and throughput between Blazegraph and QLever.
Moreover, we will be able to observe QLever's memory allocation and thread utilization patterns under load and with the system in steady state.
The test plan is roughly as follows:
- Rewrite wikidata_main queries in batch, preserving the event produced timestamp.
- Produce rewritten queries from HDFS to kafka-test with query and meta.dt fields ({"meta": {"dt": <rewritten_dt>}, "query": <rewritten_query>}).
- Replay traffic deterministically via queryhammer to internal WDQS routes (queries won't leave WMF network perimeter). The target will be wikidata_main. We will replay the whole Blazegraph fleet on the 6 wdqs-main replicas currently deployed in DPE.
- Collect and report queryhammer stats.
- Collect and report Prometheus metrics for WDQS v2 endpoints.
Control
- We need to test both internal and external endpoint queries
- We compared performance by submitting queries via the proxy vs. directly against QLever.
Related WIP
Other tasks in flight, with different (but related) angles on perf measurement.
AC
- we have provided upper/lower throughput bounds to inform rate limit choices


