Page MenuHomePhabricator

Editor analytics report for Android app (language)
Closed, ResolvedPublic

Description

  • # of title description new editors across languages (i.e. accounts whose first edit on any wiki is a Wikidata description edit using the Android app)
  • # of image caption new editors across languages (i.e. accounts whose first edit on any wiki is an image caption edit using the Android app)
  • # of image tag new editors across languages (i.e. accounts whose first edit on any wiki is an image tag edit using the Android app)
  • Proportion of all SE edits in each language (e.g., 50% of all SE edits were in English, 10% in Hindi, etc...)
  • Proportion of individual types of SE edits in each language (i.e., 50% of image captions were in English, 10% in Hindi, etc..)

We will want to run this report at regular intervals to measure our effectiveness, so if this can be a dashboard it would be useful.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 3 2018, 5:58 PM
MBinder_WMF triaged this task as Medium priority.Apr 26 2018, 8:20 PM
MBinder_WMF moved this task from Triage to Backlog on the Product-Analytics board.

This needs some clarification. The task title suggests this is about all kinds of edits made with the app, but the task description and the linked doc are about the particular case of Wikidata description edits. For that case, we should build on the research done last year, which already answered question #2 (revert rates) and can be easily extended in various ways.

Since Android doesn't really cater for any editing except Wikidata descriptions, the information requested is about those. :)

Actually, non-Wikidata edits still make up the vast majority of the edits that have been done with the app so far.
(For a quick check, compare the stats for the "mobile app edit" tag on Wikidata (167,102 changes changes right now) and on English Wikipedia (1,531,407 changes). While the latter includes iOS, there is no question that the normal edit button for article text has seen much more usage at this point on the Android app than the description editing feature. Can discuss this in more detail when we talk later today.)

Cool. Happy to have stats for both to compare. I don't think we cater super well for other editing right now, but at the very least if we know what we have then it's a good baseline for seeing if we've actually helped people when we introduce more editing features next FY.

Tbayer renamed this task from Editing behaviour on Android app to Wikidata editing behaviour on Android app.May 10 2018, 3:50 AM
Tbayer updated the task description. (Show Details)

Clarified the scope of this task per @Charlotte, and split off the question about the volume of general edits into T194424.
I also updated my analysis of reverts for description edits for the last few months, addressing question 2 in this task. (This consisted just of re-running the existing PAWS notebook and thus wasn't much work at all, but still took a while as the calculation kept failing because - I think - of time limits on PAWS; the revert analysis is a bit computationally expensive and took 8 hours when it finally completed successfully; next time we run this one probably needs to invest some time to split it into several shorter timespans).

nshahquinn-wmf added a subscriber: nshahquinn-wmf.EditedMay 10 2018, 9:55 PM

The calculation kept failing because - I think - of time limits on PAWS; the revert analysis is a bit computationally expensive and took 8 hours when it finally completed successfully; next time we run this one probably needs to invest some time to split it into several shorter timespans.

Have you considered using SWAP instead? I've never run into resource limitations like those, probably because the service is optimized for a small number of heavy users rather than a large number of light to moderate users.

The downside is probably that it's a bit harder to make your work public (I'm not sure because I've never used PAWS), but I've been able to address that by uploading my notebooks to GitHub.

In T184096#4198854, @Neil_P._Quinn_WMF wrote:

The calculation kept failing because - I think - of time limits on PAWS; the revert analysis is a bit computationally expensive and took 8 hours when it finally completed successfully; next time we run this one probably needs to invest some time to split it into several shorter timespans.

Have you considered using SWAP instead? I've never run into resource limitations like those, probably because the service is optimized for a small number of heavy users rather than a large number of light to moderate users.

That may be a good idea, although the limiting factor here is not database load or computations in Python but the large number of web API accesses. (I have been using the API version of mwreverts because last year I couldn't get the database version to work on PAWS, even with Aaron's support - filed at https://github.com/mediawiki-utilities/python-mwreverts/issues/8 .)

Restricted Application edited projects, added Product-Analytics; removed Product-Analytics (Kanban). · View Herald TranscriptOct 16 2019, 5:47 PM
LGoto assigned this task to SNowick_WMF.Apr 10 2020, 5:24 PM
LGoto moved this task from Backlog to Needs Investigation on the Product-Analytics board.

Removed the bullet point about people who micro-contribute and whether they become full editors - that is a large research project that we've captured elsewhere in our backlog: T232052

I'd like to breathe new life into this ticket, especially now that we are trying to increase account creation.

Charlotte renamed this task from Wikidata editing behaviour on Android app to Editor analytics report for Android app.Jul 15 2020, 8:53 PM
Charlotte updated the task description. (Show Details)
Charlotte raised the priority of this task from Medium to High.Jul 27 2020, 2:20 PM
Charlotte updated the task description. (Show Details)
Charlotte removed a subscriber: Tbayer.
Charlotte renamed this task from Editor analytics report for Android app to Editor analytics report for Android app (language + acct creation).Jul 27 2020, 6:43 PM
Charlotte renamed this task from Editor analytics report for Android app (language + acct creation) to Editor analytics report for Android app (language).Jul 27 2020, 6:49 PM
Charlotte updated the task description. (Show Details)
SNowick_WMF added a subscriber: Dbrant.EditedAug 6 2020, 11:25 PM

I have a dashboard started for this, how far back do we want data? Using the same data set of new Suggested Edits users as of 2020-04-01 is on Superset as Draft: https://superset.wikimedia.org/superset/dashboard/161

I need to check @Dbrant where/if image tag language language is recorded in Event Logging, because it's on Commons and goes into a Special Page designation there isn't a language code in the usual event_comment - may be able to figure out language for tags another way.

SNowick_WMF updated the task description. (Show Details)Aug 6 2020, 11:26 PM
SNowick_WMF updated the task description. (Show Details)
SNowick_WMF updated the task description. (Show Details)
SNowick_WMF updated the task description. (Show Details)Aug 6 2020, 11:29 PM
SNowick_WMF updated the task description. (Show Details)
Charlotte lowered the priority of this task from High to Medium.Aug 12 2020, 5:57 PM
SNowick_WMF updated the task description. (Show Details)Sep 28 2020, 8:15 PM
SNowick_WMF updated the task description. (Show Details)
SNowick_WMF updated the task description. (Show Details)Sep 28 2020, 8:20 PM

Was waiting for image tag data but since they are Wikidata items and are translated by Wikidata the entry language is not tracked. Dashboard ends at last month since we use mediawiki_history to collect this aggregated data.

https://superset.wikimedia.org/superset/dashboard/androidlanguagesuggestededits/

@Charlotte I know you had an additional request re. charts and isolating them by language, did you still want me to make those charts? If so please add details, I believe it was for all image captions (or other type edit) vs SE image captions (or other type SE edit) by language for the top 10 languages.

SNowick_WMF raised the priority of this task from Medium to High.Sep 28 2020, 8:31 PM
SNowick_WMF moved this task from Next 2 weeks to Needs Review on the Product-Analytics (Kanban) board.

Was waiting for image tag data but since they are Wikidata items and are translated by Wikidata the entry language is not tracked. Dashboard ends at last month since we use mediawiki_history to collect this aggregated data.

https://superset.wikimedia.org/superset/dashboard/androidlanguagesuggestededits/

@Charlotte I know you had an additional request re. charts and isolating them by language, did you still want me to make those charts? If so please add details, I believe it was for all image captions (or other type edit) vs SE image captions (or other type SE edit) by language for the top 10 languages.

Honestly, @SNowick_WMF, I think we're good here. This is more than enough to go on. Cheers!

Charlotte closed this task as Resolved.Sep 28 2020, 9:19 PM