Page MenuHomePhabricator
Create Task
Maniphest T225911

Add new mediatypes to media classification refinery code
Closed, ResolvedPublic5 Estimated Story Points
Actions

Description

At the moment it's only registering a small number of file types, so the file classification is not great.

Details

SubjectRepoBranchLines +/-
[wip]Add file extension and media classification to mediacounts jobanalytics/refinerymaster+19 -11
Add file extension and media type classification to media files UDFanalytics/refinery/sourcemaster+561 -77
Customize query in gerrit

Related Objects
Search...

Event Timeline

• fdans created this task.Jun 17 2019, 9:10 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 17 2019, 9:10 AM
Ottomata assigned this task to • fdans.Jun 17 2019, 3:37 PM
Ottomata triaged this task as High priority.
Ottomata moved this task from Incoming to Data Quality on the Analytics board.
Ottomata added a project: Analytics-Kanban.
• fdans moved this task from Next Up to In Code Review on the Analytics-Kanban board.Jun 18 2019, 1:19 PM
gerritbot added a comment.Jun 18 2019, 1:19 PM

Change 517641 had a related patch set uploaded (by Fdans; owner: Fdans):
[analytics/refinery/source@master] Add media formats to file url parser regex

https://gerrit.wikimedia.org/r/517641

gerritbot added a project: Patch-For-Review.Jun 18 2019, 1:19 PM
• Nuria subscribed.EditedJun 19 2019, 7:40 PM

@fdans let's please file tasks for mediarequest api as child tasks of https://phabricator.wikimedia.org/T207208 so we keep track of all the work we are doing on this regard.

• Nuria added a parent task: T207208: Add mediacounts data to AQS and, from there, Restbase.Jun 19 2019, 7:40 PM
gerritbot added a comment.Jul 12 2019, 9:39 AM

Change 522390 had a related patch set uploaded (by Fdans; owner: Fdans):
[analytics/refinery@master] [wip] Add file extension and media classification to mediacounts job

https://gerrit.wikimedia.org/r/522390

• fdans added a parent task: T228149: Load media requests data into cassandra .Jul 16 2019, 11:21 AM
• fdans removed a parent task: T207208: Add mediacounts data to AQS and, from there, Restbase.
gerritbot added a comment.Jul 23 2019, 11:02 AM

Change 517641 merged by Fdans:
[analytics/refinery/source@master] Add file extension and media type classification to media files UDF

https://gerrit.wikimedia.org/r/517641

• Nuria renamed this task from Expand regex that maps file types to media to Add new mrediatypes to media classification refinery code.Jul 23 2019, 2:48 PM
• Nuria moved this task from In Code Review to Ready to Deploy on the Analytics-Kanban board.
• Nuria renamed this task from Add new mrediatypes to media classification refinery code to Add new mediatypes to media classification refinery code.Jul 24 2019, 5:00 PM
• Nuria moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Aug 1 2019, 3:33 PM
• Nuria added a comment.Aug 1 2019, 7:15 PM

In order for the code to take effect the job needs to be re-started

gerritbot added a comment.Aug 7 2019, 3:28 PM

Change 522390 abandoned by Fdans:
[wip]Add file extension and media classification to mediacounts job

Reason:
Since we'll be using a new dataset for the mediarequests API, I'm going to abandon this and open a new change instead

https://gerrit.wikimedia.org/r/522390

Maintenance_bot removed a project: Patch-For-Review.Aug 7 2019, 4:10 PM
• Nuria closed this task as Resolved.Sep 19 2019, 6:30 PM
• Nuria set the point value for this task to 5.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL