Background
We want to measure the capability of users to federate queries to different endpoints.
Possible metrics for this could be, for example:
- # of other endpoints users successfully federated to
- % of subqueries that failed because the target endpoint was not allowlisted
Task
Establish a logic for the guiding metric that we can use and take monthly baselines from the data available.
Acceptance Criteria
A script that can be run on a certain graph for a chosen period of time that returns a value for the metric or a list of values that the metric can be calculated from
Notes
We previously learnt to measure the amount of attempts to use the SERVICE syntax in the parsed queries.
If we build on this approach, we would need to cluster each row in this query into:
- federation attempt to an allowlisted endpoint (success)
- federation attempt to an allowlisted endpoint (failed)
- federation attempt to non-allowlisted endpoint
- self-federation attempt (federation to the same graph where the query is running on)
- non-federation (invoking a helper service)
- could not cluster.
The tickets to make such clustering possible are: T399805, T399827, T399828, T399831
Another approach would also be to analyze the actual logs of outgoing subqueries and cluster those (in this case we will not need the 'non-federation' category, because no subquery is being produced in this case).
A single ticket was created for this investigation: T399829