Maniphest T360523

Refactor Page Analytics according to the new AQSAssist
Open, Needs TriagePublic
Actions

Assigned To

Authored By

	JEbe-WMF
	Mar 20 2024, 2:10 PM

Tags

Referenced Files

None

Subscribers

EChukwukere-WMF

VirginiaPoundstone

Description

Goal

Extract to aqsassist some repeated and common code that is present in all the services and refactor them to avoid code duplication as we already did with the functions we created to validate the parameters in the handlers.

About the Druid common code refactoring we will take the following into consideration:

The way to parameterize Druid queries that was implemented in AQS 1.0 (https://github.com/wikimedia/analytics-aqs/blob/master/sys/mediawiki-history-schemas.yaml) to parametrize Druid queries using a separate file
We'll start refactoring editor-analytics because its code is already done and we'll have the opportunity of using its unit and integration tests to test the refactored service.

Tasks

Refactor
QA test
Deploy

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		JEbe-WMF	T343907 AQS 2.0: aqsassist - Extract some repeated common code from services to this module
		Open		JEbe-WMF	T360523 Refactor Page Analytics according to the new AQSAssist

Event Timeline

JEbe-WMF created this task.Mar 20 2024, 2:10 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 20 2024, 2:10 PM

JEbe-WMF moved this task from Sprint Backlog to Code Review / Tech Input on the Data Products (Data Products Sprint 10) board.Mar 20 2024, 2:14 PM

JEbe-WMF moved this task from Code Review / Tech Input to Paused on the Data Products (Data Products Sprint 10) board.Mar 20 2024, 4:15 PM

WDoranWMF moved this task from Paused to Protal to Sprint 11 on the Data Products (Data Products Sprint 10) board.Mar 21 2024, 2:01 PM

WDoranWMF edited projects, added Data Products (Data Products Sprint 11); removed Data Products (Data Products Sprint 10).

WDoranWMF moved this task from Sprint Backlog to Paused on the Data Products (Data Products Sprint 11) board.

JEbe-WMF moved this task from Paused to Testing on the Data Products (Data Products Sprint 11) board.Wed, Apr 17, 10:02 AM

JEbe-WMF edited projects, added Data Products (Data Products Sprint 12); removed Data Products (Data Products Sprint 11).Wed, Apr 17, 2:11 PM

JEbe-WMF moved this task from Sprint Backlog to Paused on the Data Products (Data Products Sprint 12) board.

JEbe-WMF moved this task from Paused to Testing on the Data Products (Data Products Sprint 12) board.Wed, Apr 17, 2:13 PM

JEbe-WMF reassigned this task from JEbe-WMF to EChukwukere-WMF.Wed, Apr 17, 2:15 PM

The changes here have been merged to prod. This has been tested through regression run for Page analytics and no issues found. All tests passed

EChukwukere-WMF moved this task from Testing to To Deploy on the Data Products (Data Products Sprint 12) board.Wed, Apr 24, 2:56 PM

EChukwukere-WMF reassigned this task from EChukwukere-WMF to JEbe-WMF.Wed, Apr 24, 3:03 PM

EChukwukere-WMF subscribed.

xcollazo moved this task from To Deploy to Testing on the Data Products (Data Products Sprint 12) board.Wed, Apr 24, 4:27 PM

Initially this was thought to be merged in production, hence wrongly tested. This is merged in the QA env and has been tested.

PROD url to test QA env with, top : "https://wikimedia.org/api/rest_v1/metrics/pageviews/top/en.wikipedia/all-access/2020/02/22"

PROD url to test QA env with, top-by-country : "https://wikimedia.org/api/rest_v1/metrics/pageviews/top-by-country/en.wikipedia.org/all-access/2020/01"

PROD url to test QA env with, aggregate : "https://wikimedia.org/api/rest_v1/metrics/pageviews/aggregate/en.wikipedia.org/all-access/all-agents/daily/2020011201/2020011518"

Local env version, top: "http://localhost:8093/metrics/pageviews/top/en.wikipedia/all-access/2020/02/22"

Local env version, top-by-country: "http://localhost:8093/metrics/pageviews/top-by-country/en.wikipedia.org/all-access/2020/01"

Local env version, aggregate: "http://localhost:8093/metrics/pageviews/aggregate/en.wikipedia.org/all-access/all-agents/daily/2020011201/2020011518"

data from prod and local were semantically identical.

No regression issues and ready for deployment.

EChukwukere-WMF moved this task from Testing to To Deploy on the Data Products (Data Products Sprint 12) board.Sun, Apr 28, 4:30 AM