Page MenuHomePhabricator

Report WDQS update latency when displaying/serving results
Open, MediumPublic

Description

As a WDQS user I want to know if the results I see come from a server that is potentially lagged so that I can evaluate if they are stale.

Currently the WDQS UI reports a latency metric by querying the backend for the following triple:

select ?lastUpdated where { <http://www.wikidata.org> schema:dateModified ?lastUpdated }

This technique assumes couple things that might not be true:

  • the server this metric is obtained from is the same than the one the results are fetched
  • the latency is stable without out of order events

Solving the first assumption might possible by changing how the UI presents this data to the user and instead of decoupling the display of this metric and the display of the results it could fetch this data from a HTTP header set by the WDQS backend.

The WDQS backend could, with an additional set of servlet filters, maintain this latency metric in memory.
The wdqs updater would set a custom header with the latency information that the filter would use to update its memory structure.
The wdqs backend would set a custom header with this latency information it has in memory.
The UI would compare the latency information with the client's "current time" giving a rough estimate of whether or not the results might be stale according to user expectations.

This solution works for cached responses as well.

Solving the second point is more difficult but using the average of the event times found in the batch being currently persisted might help.

AC:

  • The WDQS UI reports the latency it finds in the response
  • The WDQS UI no longer reports latency information that is not related to the server serving the results
  • The latency information is no longer stored in the triple store
  • The WDQS backend reports latency information from a custom HTTP header X-Last-Event-Time (name TBD)
  • The WDQS backend updates its latency information from a custom HTTP header X-Set-Last-Event-Time on write requests
  • The event time is an average of the batch being processed