Page MenuHomePhabricator

Provide edit samples and user lists for mobile contribution design research
Closed, ResolvedPublic


Product-Design-Strategy is starting a research project about mobile contribution.

They've identified the following needs:

  • Overall data on how many only-mobile and partly-mobile editors there are. A histogram of editor count by mobile edit proportion is probably the right deliverable.
  • Random samples of 200 mobile edits from each of the English, Czech, Korean, French, and Hindi Wikipedias for manual analysis.
    • On each wiki, 160 should be mobile web edits, 20 should be Android edits, and 20 should be iOS edits.
  • A selection of users Abbey can email for the survey. The population is (1) English Wikipedia editors (2) with email addresses (3) who have made at least 5 edits (4) who have edited in the past 90 days. The stratification by region, platform, and experience is described in this spreadsheet. The data should be:
    • separated into two files based on experience
    • tagged with user name, exact edit count, and user age
    • uploaded into Qualtrics for @aripstra and @egalvezwmf to use.

Event Timeline

Restricted Application changed the subtype of this task from "Deadline" to "Task". · View Herald TranscriptSep 12 2018, 10:39 PM
nshahquinn-wmf moved this task from Triage to Doing on the Product-Analytics board.
nshahquinn-wmf moved this task from Doing to Next Up on the Product-Analytics board.

@aripstra, @MNovotny_WMF, it looks to me like I can already work on the first two deliverables, but I'm blocked on the recruit lists until you decide the strategy (which may, of course, depend on the data on mobile contribution rates).

On a related note, do you know how you're going to reach out to these contributors? I'd suggest email, because our selection is based partly on these users' geographic home countries, so posting an invitation on their talk page has the potential to out their general location. That will reduce the pool of possible invitees by about half, since users don't have to give an email address to sign up, but we should still have enough unless you need, say, 20 editors from Azerbaijan 😁

@Neil_P._Quinn_WMF, Great to hear you can do the first two. I wrote up a draft recruitment strategy (the third bullet) and have shared it with you. We can spend a few mins of our next 1:1. I think you are correct that the data on mobile contribution rates will inform this recruitment strategy. :)

I am planning, at this point, on emailing people to invite them to participate in the survey. I, of course, do not want to take the risk of outing people's locations.

Related to the second bullet: "Random samples of 200 mobile edits from four different wikis for manual analysis.", There was a suggestion to pull 200 random edits from more than the 5 Wikipedias I had originally scoped it to (which are En, Ko, Cz, Fr and Hi). This is because there is already a process set up for tagging edits, and people might volunteer to tag edits from their Wikipedias. This way, we could learn about mobile edits from many more wikis without expanding the scope of the work of this specific project, by working with volunteers to tag more edits. Whichever sets of edits get tagged, we will know more about that wiki.

So, my question to you, is : How many wikipedias could you pull 200 random edits from without making this too much more work? (I remember you saying pulling more edits is not too much more work.)

@Neil_P._Quinn_WMF After a few conversations, and some thinking, I think pulling 200 random mobile edits from these wikis (English, Korean, Czech, Hindi and French) will be sufficient for this project.

Another possible project for after this one is completed, could be a larger mobile edit tagging project. But for this work, I would like to keep the scope tighter. So, I rescind asking that last question to you.

Neil and I talked, and we agreed that he will provide the edit samples by the end of day (pacific time) Tuesday, October 30th. Thank you Neil!!!

Okay, @aripstra, I've pulled the edit samples! They are in this spreadsheet, with a separate tab for each wiki.

The only information I've provided is the platform, the revision ID, and a link to the diff page for the edit. I actually wanted to add the user name and page title too but that turned out to be surprisingly complex so I skipped it. If this becomes a major issue, I could potentially go back and provide it.

These are randomly sampled from all the edits made from August to October using the given platform and on the given wiki; in some cases there were very few edits in that bucket, so for example our 20 Hindi Wikipedia iOS edits were only pulled from a population of 31!

       android app edit ios app edit mobile web edit
cswiki              508           95            8252
enwiki            54249        32110         1023227
frwiki             8326         3573          106010
hiwiki             1187           31           14130
kowiki             5272         2283           51167

I will do the editor lists next and will have them by the start of the next week.

Thank you for the edit samples, @Neil_P._Quinn_WMF. They are just what we needed. No worries about the username and page title not being in the list. We can see that in the edit itself, and don't need it to be part of the higher level info.

I've now uploaded the survey lists to Qualtrics, so we can call this done.

Code is archived in the repo [wikimedia-research/2018-10-Design-Research-mobile-research-samples](