⚓ T244373 Fix double encoding of urls on mediarequests api

	Subject	Repo	Branch	Lines +/-
	Handle punctuation chars in paths for mediarequests per file	analytics/aqs	master	+54 -18
	Url encode file name before querying the data store	analytics/aqs	master	+33 -9

Status	Assigned	Task
Resolved	None	T210313 Statistics for views of individual Wikimedia images
Resolved	MusikAnimal	T234590 Add ability to the pageview tool in labs to get mediarequests per file similar to existing functionality to get pageviews per page title
Resolved	• fdans	T244373 Fix double encoding of urls on mediarequests api

• Nuria created this task.Feb 5 2020, 4:05 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 5 2020, 4:05 PM

• Nuria reassigned this task from MusikAnimal to • fdans.Feb 5 2020, 5:06 PM

• Nuria added a project: Analytics-Kanban.

• Nuria updated the task description. (Show Details)

• Nuria updated the task description. (Show Details)Feb 5 2020, 5:11 PM

Change 570608 had a related patch set uploaded (by Fdans; owner: Fdans):
[analytics/aqs@master] Url encode file name before querying the data store

https://gerrit.wikimedia.org/r/570608

gerritbot added a project: Patch-For-Review.Feb 6 2020, 10:57 AM

• fdans triaged this task as Unbreak Now! priority.Feb 6 2020, 6:08 PM

• fdans moved this task from Incoming to Analytics Query Service on the Analytics board.

• fdans moved this task from Analytics Query Service to Radar on the Analytics board.

• fdans moved this task from Radar to Analytics Query Service on the Analytics board.

Restricted Application added a subscriber: Liuxinyu970226. · View Herald TranscriptFeb 6 2020, 6:08 PM

Harej unsubscribed.Feb 11 2020, 1:16 AM

• fdans moved this task from Next Up to In Code Review on the Analytics-Kanban board.Feb 12 2020, 8:53 AM

Change 570608 merged by jenkins-bot:
[analytics/aqs@master] Url encode file name before querying the data store

https://gerrit.wikimedia.org/r/570608

@MusikAnimal FYI this change will be deployed tomorrow

Maintenance_bot removed a project: Patch-For-Review.Feb 19 2020, 7:10 PM

Ok, I deployed this, seems to fix the original problem as reported in T234590#5850051, as now it allows the single-encoded form:

https://wikimedia.org/api/rest_v1/metrics/mediarequests/per-file/all-referers/user/%2Fwikipedia%2Fcommons%2F5%2F53%2FPr%C3%A4sidentschaftswahl_in_den_Vereinigten_Staaten.ogv/daily/2019020100/2020013100

However, it no longer allows the double encoded form, so I'm guessing this broke @MusikAnimal's interface until he turns off the hack.

• fdans moved this task from In Code Review to Done on the Analytics-Kanban board.Mar 3 2020, 1:45 PM

Thanks for the fix (and apologies for my delayed response)! Should this be closed? It does indeed work in Mediaviews now https://tools.wmflabs.org/mediaviews/?project=commons.wikimedia.org&platform=&referer=all-referers&range=latest-20&files=Pr%C3%A4sidentschaftswahl_in_den_Vereinigten_Staaten.ogv

There are a few outstanding issues with the mediarequests data that we need to solve for next quarter but this very one can be closed, thanks.

Change 588396 had a related patch set uploaded (by Fdans; owner: Fdans):
[analytics/aqs@master] Handle punctuation chars in paths for mediarequests per file

https://gerrit.wikimedia.org/r/588396

gerritbot added a project: Patch-For-Review.Apr 13 2020, 12:42 PM

I've documented the few different variations of file paths here:
https://wikitech.wikimedia.org/wiki/Analytics/AQS/Mediarequests/Limitations#Issues_with_file_paths

Change 588396 merged by jenkins-bot:
[analytics/aqs@master] Handle punctuation chars in paths for mediarequests per file

https://gerrit.wikimedia.org/r/588396

Maintenance_bot removed a project: Patch-For-Review.Apr 27 2020, 9:10 PM

Nice docs, thanks for taking the time @fdans Ping to @elukey to keep in mind whether the ATS will encode urls on the same manner

• Nuria closed this task as Resolved.Apr 30 2020, 4:18 PM

MusikAnimal moved this task from Backlog to Done on the Tool-Pageviews board.Feb 12 2021, 12:00 AM