Page MenuHomePhabricator
Create Task
Maniphest T431814

🧹 [Random Sampling] Some automated categorization of our Wikidata RC sample using non-diff data sources (🏆)
Open, Needs TriagePublic
Actions

Description

We already know about some important, major categories of RC diff such as "Language link added" which are expected to not show up in our visible diffs because they change (in the example of language link) sitelinks or something else meaningful about the page.

This might involve running a pyspark module on the analytics cluster to gather this information, if it wasn't provided by the recentchanges API.

Be careful not to pull any private data into our results files. Aggregate before storing (eg. to a boolean "is language link"), if there's any risk.

Related Objects
Search...

Event Timeline

awight created this task.Fri, Jul 10, 11:20 AM
awight moved this task from Needs Refinement to Sprint Backlog (This Sprint, Ready to pick up) on the Wikidata Integration in Wikimedia projects (Kanban Board) board.Mon, Jul 13, 8:24 AM
Lucyfediachambers renamed this task from Some automated categorization of our Wikidata RC sample using non-diff data sources to 🧹 Some automated categorization of our Wikidata RC sample using non-diff data sources.Mon, Jul 13, 8:26 AM
Lucyfediachambers renamed this task from 🧹 Some automated categorization of our Wikidata RC sample using non-diff data sources to 🧹 [Random Sampling] Some automated categorization of our Wikidata RC sample using non-diff data sources.Mon, Jul 13, 9:00 AM
Ifeatu_Nnaobi_WMDE subscribed.Mon, Jul 13, 2:34 PM

Is there a way to see further categories that are not only language link added so that we can find out the various proportions of other potentially annoying notifications that don't create a diff

seanleong-WMDE claimed this task.Tue, Jul 14, 8:09 AM
seanleong-WMDE moved this task from Sprint Backlog (This Sprint, Ready to pick up) to In Progress (+ sprint emoji to the title on entry) WIP = 4 on the Wikidata Integration in Wikimedia projects (Kanban Board) board.
Lucyfediachambers renamed this task from 🧹 [Random Sampling] Some automated categorization of our Wikidata RC sample using non-diff data sources to 🧹 [Random Sampling] Some automated categorization of our Wikidata RC sample using non-diff data sources (🏆).Tue, Jul 14, 8:13 AM
Nicholusmuwonge_wmde subscribed.Tue, Jul 14, 9:21 AM
In T431814#12115114, @Ifeatu_Nnaobi_WMDE wrote:

Is there a way to see further categories that are not only language link added so that we can find out the various proportions of other potentially annoying notifications that don't create a diff

Hey Ifeatu, do you have any that come to mind?

seanleong-WMDE removed seanleong-WMDE as the assignee of this task.Wed, Jul 15, 12:48 PM
seanleong-WMDE moved this task from In Progress (+ sprint emoji to the title on entry) WIP = 4 to Sprint Backlog (This Sprint, Ready to pick up) on the Wikidata Integration in Wikimedia projects (Kanban Board) board.
seanleong-WMDE subscribed.
Lucyfediachambers moved this task from Sprint Backlog (This Sprint, Ready to pick up) to Refined (ready for future sprints) on the Wikidata Integration in Wikimedia projects (Kanban Board) board.Wed, Jul 29, 10:03 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits