Page MenuHomePhabricator

[REQUEST] Android Data Analysis - Want to see if gatekeeping minimum decreases revert rate for image captions
Closed, DuplicatePublic


Per @Charlotte

What's requested:
Want to see if gatekeeping minimum decreases revert rate for image captions
Users used to have to write a minimum of X article descriptions before we allowed them to write captions
Likewise a minimum number of added descriptions/captions before we allowed ppl to translate things
Now users are given access to all SE tasks at the same time
Want to see if removal of minimum number of article descriptions increases revert rate of captions

Why it's requested:
Data Analysis
When it's requested:

Other helpful information:

Event Timeline

SNowick_WMF moved this task from Triage to Needs Investigation on the Product-Analytics board.

@Charlotte Let's discuss priority for this and I will determine where data is pulled from.

Compare revert rates from pre-11/25 and post feature removal for logged in users
Is gatekeeping is effective?
Translate captions

per @mpopov

basically the easiest/only way you could do this is:
to know which app users made which edits grab the wiki + app install ID + revision ID tuples from MobileWikiAppEdit table where action = 'saved'

to know the revert status of those edits grab wiki + revision ID + revert_is_identity_reverted tuples from mediawiki_history where array_contains(revision_tags, 'android app edit')

to know who’s new to the app grab the app install IDs from MobileWikiAppInstallReferrer, but that’s known to not fire 100% of the time so you’d probably want to augment with app install IDs from MobileWikiAppDailyStats (with appInstallAgeDays < N and appropriate N for year, month) which is ALSO not known to fire 100% of the time it should (as we’ve seen recently)

do an inner join on everything because you’re bound to get some IDs in every table that aren’t in any of the other tables. whatever’s left is your best estimate of revert rate for users new to the app

this comes at the massive caveat that there may be some users in the mix who HAVE used the app before but maybe uninstalled & reinstalled it or maybe they’ve switched devices, but in any case we don’t keep track of who’s completely new to the app and who’s “new” but returning

Revisions to Commons aren't tracked on 'mobilewikiappedit' so this needs more work, we may not be able to lineup app version in wherever edits are tracked.