Request details
- Getting this request in early in the process so we're on top of this and it can be prioritized as needed
- WMDE would like to release metrics on certain types of queries that are being made against the Wikidata Query Service
- For now the data would go to the published datasets directory for WMDE Analytics as CSV files that are a select all from the resulting table of an Airflow DAG
- We'd eventually like this to be displayed in a public dashboard, but this is later
- Data would include the total queries for a period as well as total queries that match certain criteria (see below)
- Assumption is that since this is user data, we need to have the select statement that creates the check to see if any of the resulting values are <25 and replace them with "<25" if that's the case
- We've done something similar in T365699
- Related deployment tasks are T370851 and T370854
Questions
Are you collecting and/or sharing data (with volunteers, grantees, or other external third party or the general public)?
- Yes
Who currently has access to this data?
- The DAG currently isn't up and running, so no one
- Without an open release of the data it would be WMF/WMDE employees who have data lake access
How long are you retaining data or planning on retaining data?
- There is no plan to set a retention limit
Are you planning on using data in a new way or collecting more data than we already do?
- Yes
Describe what the data is, how it will be used, and any technical integrations required to use it.
- An output of the data would be along the lines of:
| day | total_wdqs_queries | total_single_entity | total_all_entity_statements | total_single_term_statement | total_single_inverse_statement | total_single_instance_or_subclass_statement | total_single_known_relation_statement | total_single_unknown_relation_statement | total_complex_queries |
|---|---|---|---|---|---|---|---|---|---|
| datetime | int | int | int | int | int | int | int | int | int |
The columns above are:
- day: The day for which the metrics are computed over
- total_wdqs_queries: Total Wikidata Query Service queries during the period
- total_single_entity: Total queries that access only one entity
- total_all_entity_statements: Total queries that access all statements of one entity
- total_single_term_statement: Total single statement queries accessing labels, aliases or descriptions
- total_single_inverse_statement: Total single statement queries that derive an entity from a predicate and an object
- total_single_instance_or_subclass_statement: Total single statement queries that query instances or subclasses of an entity subject
- total_single_known_relation_statement: Total single statement queries that check for an known relationship between a subject and an object
- total_single_unknown_relation_statement: Total single statement queries that check for an unknown relationship between a subject and an object
- total_complex_queries: Total queries not included in any custom aggregation column
Is this request urgent?
- Yes, we need these metrics for the collaboration agreement