Goal
Collect all inconsistencies found in AQS 1.0 services and some existing issues in AQS 2.0 services for possible future improvements
Current issues
- Date range: [Breaking change] Which range specify a full month? e.g.: 20230501/20230601 vs 20230501/20230531 between edit-analytics and device-analytics. "One month" has different considerations depending on the service.
{
"detail": "no full months found in specified date range",
"method": "get",
"status": 400,
"title": "Bad Request",
"type": "about:blank",
"uri": "/metrics/edits/aggregate/ab.wikipedia/user/content/monthly/20200201/20200229"
}{
"items": [
{
"project": "en.wikipedia",
"access-site": "all-sites",
"granularity": "monthly",
"timestamp": "20200201",
"devices": 875114780,
"offset": 309778523,
"underestimate": 565336257
}
]
}- Response structure conventions: [Breaking change] e.g.: lower_snake_case vs kebab-case between media-analytics & editor-analytics (sometimes even in the same service: editor/top-by-edits). Some services use lower_snake_case and others kebab-case. Some times even both conventions can be considered in the same response for the same service
{
"items": [
{
"referer": "all-referers",
"media_type": "all-media-types",
"agent": "all-agents",
"granularity": "monthly",
"timestamp": "2020010100",
"requests": 77922346640
},
{
"referer": "all-referers",
"media_type": "all-media-types",
"agent": "all-agents",
"granularity": "monthly",
"timestamp": "2020020100",
"requests": 74434493109
}
]
}{
"items": [
{
"project": "ab.wikipedia",
"editor-type": "user",
"page-type": "non-content",
"activity-level": "1..4-edits",
"granularity": "daily",
"results": [
{
"timestamp": "2023-04-01T00:00:00.000Z",
"editors": 0
},Error response structure: [Breaking change] [Done] e.g.: detail[] vs detail property (this issue was already discussed and fixed. detail field is no longer an array in AQS 2.0). We don't have to do anything because it's already considered and fixed in all the AQS 2.0 services. See T302536: Problem details for HTTP APIs (rfc7807)
{
"type": "https://mediawiki.org/wiki/HyperSwitch/errors/bad_request",
"title": "Invalid parameters",
"method": "get",
"detail": "data.params['page-type'] should be equal to one of the allowed values: [all-page-types, content, non-content]",
"uri": "/wikimedia.org/v1/metrics/editors/aggregate/ab.wikipedia/user/non-constent/1..4-edits/daily/2024040112/2024050112"
}{
"type": "https://mediawiki.org/wiki/HyperSwitch/errors/invalid_request",
"method": "get",
"detail": [
"end timestamp is invalid, must be a valid date in YYYYMMDD format"
],
"uri": "/analytics.wikimedia.org/v1/mediarequests/aggregate/all-referers/all-media-types/all-agents/daily/20200101/2023053000a"
}No data found error: [Breaking change] [Done] e.g.: empty 200 OK vs 404 between edit-analytics & device-analytics. Druid-based services consider an empty results array as a valid response for non-existent projects. Cassandra ones respond with a 404 Error. Both cases could be consider as ok but we should decide the same for all the services. In the end we decided to change how Druid-based services behaved and now all services are throwing a 404 Not Found error when a non-existent project is requested.
{
"items": [
{
"project": "ab.wikipedia",
"editor-type": "user",
"page-type": "content",
"granularity": "daily",
"results": []
}
]
}{
"type": "https://mediawiki.org/wiki/HyperSwitch/errors/not_found",
"title": "Not found.",
"method": "get",
"detail": "The date(s) you used are valid, but we either do not have data for those date(s), or the project you asked for is not loaded yet. Please check https://wikimedia.org/api/rest_v1/?doc for more information.",
"uri": "/analytics.wikimedia.org/v1/unique-devices/en.wikipedia.org/all-sites/daily/20240101/20240101"
}- Removing artificial zero-values: AQS 1.0 Druid services, when any data is found for the current request, put an artificial zero-value when there is no data for a specific date. After discussing that, we decided not to include those zero-values in AQS 2.0 druid services responses and leave empty the results for this specific date (TODO: add a sample here)
- Documentation: [Non breaking change] kebab-case vs lower_snake_case when specifying path parameters in some services (see AQS 1.0 OpenAPI Spec). This inconsistency only affects to the OpenAPI specification document and it's being fixed while writing the docs for new AQS 2.0 services.
- Documentation: [Non breaking change] How to use all-projects and all-[family]-projects keywords is not always clear. Some endpoints explain explicitly you can use it and others don't but you can really use it. For example: it can be used when requesting edits/per-page/ab.wikipedia/Абжьыуа_жәлақәа/all-editor-types/daily/20230301/20230601 but it's not explained in the documentation for that endpoint.
Projects in personal repos: [Refactoring task] Some AQS related projects are hosted in personal repos. We should move them to a non-personal one:- aqsassist https://gitlab.wikimedia.org/frankie/aqsassist
- aqs-docker-test-env https://gitlab.wikimedia.org/frankie/aqs-docker-test-env
- aqs-druid-test-env: https://github.com/bpirkle/aqs-docker-druid-test-env
- Lack of data in mediarequest_top_files: There is a lack of data for the following dates: 2019.3, 2019.4 and 2019.5 when using the "all-days" value as "day" in mediarequest_top_files dataset (media-analytics). For instance: https://wikimedia.org/api/rest_v1/metrics/mediarequests/top/all-referers/all-media-types/2019/03/all-days
- Prepare device-analytics to be dockerized and add it to the aqs-docker-test-env-qa: [QA improvement] Starting from geo-analytics, we made some changes to every service's repository to be able to dockerize them and run the QA test suite using docker (some changes were done as well in aqs-docker-test-env project to create a docker-compose project with cassandra and all its services). Device analytics was deployed before that so it has not been included yet.
- Automated way to test that /metrics endpoint is generating metrics properly: [QA improvement] /metrics endpoint can be present but not really working. The presence of this endpoint and real metrics generation are separate steps. Just adding the endpoint a default summary is created when you access to it, but not live metrics are shown there. We need to be sure of both things: the endpoint is present and the endpoint is generating metrics for every request
- Automated way to test that /api-spec.json endpoint is present and working: [QA improvement] For some services, integration and unit test cases for this endpoint are also missing (geo, page, edit and editor)
- Project, referer and page-title validation/normalization: [Missing validation/normalization feature] During first part of the development we missed some validation/normalization process that was done by AQS 1.0 using some common functions. This was done for page, edit and editor but it is pending for geo, device and media. We created new aqsassist functions to do these tasks and they are called via a middleware function in each service project. It could be useful to take a look at page, edit or editor services to see how was addressed for these services, and also at T346598: [Pageviews_top_by_country] requests with projects with special characters returns 404 instead of 400
- aqsassist.ValidateProject: Validate and normalize a project
- aqassist.ValidateReferer: Validate and normalize a referer
- aqsassist.NormalizePageTitle: Normalize a page-title replacing all occurrences of spaces with underscores
- aqssassist common code for messaging, problems and loggers: [Refactoring task] During AQS 2.0 development some common code was added to aqsassist to manage messages, problems and loggers (some details at T343907: AQS 2.0: aqsassist - Extract some repeated common code from services to this module). Some services were deployed before completing that work so it's something we should do later to unify all services. This work is pending review and it haven't been done for any service. Keep in mind that Druid refactoring, also included in that task, is already done for both Druid-based services. It's a good opportunity to unify messages through all the services. For example, at this moment there are three different styles for messaging:
- device-analytics detail message when wrong granularity is requested: Invalid granularity
- geo-analytics detail message when wrong activity-level is requested: Activity-level should be equal to one of the allowed values: [5..99-edits, 100..-edits]
- media-analytics (and page,edit and editor) detail message when wrong granularity is requested: granularity should be equal to one of the allowed values: [daily, monthly]
- Remove indentation from response / Minify response: [Refactoring task] All 2.0 services respond with an indented response body but traffic team warned us about that because responses were bigger than before (66% bigger in some cases). That's why we decided to remove indentation from responses for all services. Because some of them were already deployed, this work is still pending for device and geo. See T346202: AQS 2.0: Remove indentation from the response body for all services for details.
- Include versioning for aqsassist package: [Refactoring task] We should start tagging the aqsassist repository to include versioning to this package.
- all-projects and all-[family]-projects keywords validation: [Missing validation]** Only editor-analytics throws a message when these keywords are used when not allowed. The rest of the services are not validating them and they throw the generic 404 Not found message. For example:
{
"detail": "all-...` project values are not accepted for this metric",
"method": "get",
"status": 400,
"title": "Bad Request",
"type": "about:blank",
"uri": "/metrics/editors/aggregate/all-projects/name-bot/all-page-types/all-activity-levels/monthly/20210302/20220901"
}{
"detail": "The date(s) you used are valid, but we either do not have data for those date(s), or the project you asked for is not loaded yet. Please check documentation for more information.",
"method": "get",
"status": 404,
"title": "Not Found",
"type": "about:blank",
"uri": "/metrics/pageviews/top/all-projects/all-access/2019/01/01"
}