Page MenuHomePhabricator

iOS Appsee Diary Study
Closed, ResolvedPublic

Description

Survey results:

https://docs.google.com/a/wikimedia.org/presentation/d/1lZ-J7NZ4560Dp25ktAhr-yQd3U2QR06Cw6ZNVNQ-7jg/edit?usp=sharing


Planning document:

https://docs.google.com/a/wikimedia.org/document/d/1tBO_MUft8QXp9BH8gy7ZrsyRXNjAxy8xj6N21ZelkUI/edit?usp=sharing

Background

Wikipedia apps on mobile devices are relatively new, and we would like to know more about how they’re used, especially observations around navigation and the Explore feed. To do this, we are both trying out new software for gathering user data (AppSee) and following the personal iOS use habits of a group of volunteers over the course of several days, to see what we can observe and set up similar tests for mobile apps generally in the future.

Links

Recruitment survey for mobile user testing: https://drive.google.com/open?id=1zBNbAfebZ2Ultccbd1lAoInda9h7NGp8KFoGtHikL2A
On-boarding survey: https://drive.google.com/open?id=1JVZ2YuP-ALAyXcL0LcWnvdhd_2-k1NCtiEYYtOFm-9w
In-app session survey (diary entry form): https://drive.google.com/open?id=1XbWjKEPVCnJ6UcKcBk9JSlFNBf-raXx9t_IdRaU4HSg


To do before study can begin

  • Create recruitment survey for mobile user testing
  • Finalize / confirm release form with legal
  • Write of [[ explanation of study and build set-up instructions for testers | explanation of study and build set-up instructions for testers ]]
  • Create onboarding survey (questions already decided) on Google forms
  • Discuss and decide on survey links / presentation of surveys with iOS team (for onboarding survey should this occur in the feed or as part of the onboarding flow? For surveys during the study should there be a push notification?)
  • Verify if Appsee will default capture the information we want captured during each session
  • Create session survey (questions already defined)
  • Recruit 12-15 new and seasoned users from our iOS beta tester pool

To do after study has begun

  • Write-up exit survey and decide how exit survey will be disseminated

Questions

  • Can we recruit from inside the app via an in-app survey?
  • How should Appsee's data capture be verified? (Should we turn it back on on Alpha?)
  • How should surveys be disseminated during the study?

Event Timeline

Regarding the question of recruiting via in-app survey – would be great if this survey can be designed in such a way so it can be used as a general participant recruitment tool for mobile apps.

Discussed in team sync, but here's the gist:

Can we recruit from inside the app via an in-app survey?

Yes. See new subtask (T156419) for developing the in-app announcement for recruitment.

How should Appsee's data capture be verified? (Should we turn it back on on Alpha?)

Yes we can re-enable Appsee any time and then turn it back off. Just let us know when you want it on.

How should surveys be disseminated during the study?

Since this is a specialized version of the app, and we want the surveys to be VERY noticeable (or even impossible to skip) I propose we do a custom feature T156404 to pop a blocking modal based on days since install, rather than use the announcement system.

In app recruitment survey is being reviewed by the Privacy team and can be viewed here:
https://drive.google.com/open?id=1zBNbAfebZ2Ultccbd1lAoInda9h7NGp8KFoGtHikL2A

cmadeo updated the task description. (Show Details)

We have initial approval from legal to move forward with sending invites to 15 perspective participants!
There are just a few questions left before we can have the privacy statement ready:

  • @JMinor, where do study results usually get posted/shared?
  • We'll also need to define how the privacy statement is presented to participants (I'll be discussing this with Aeryn via email)

On a side note, would it be possible to have unsubscribes from the recruitment survey be handled by OTRS or another email address? Right now the plan was for unsubscribes to go to my email address, but perhaps this isn't the most sustainable solution. Thoughts welcome.

There are just a few questions left before we can have the privacy statement ready:

@JMinor, where do study results usually get posted/shared?

On mediawiki. For official research team supported projects (like the reader motivation study) there is a Research project template and space. For previous "in house" design research we've published them on Mediawiki in the Apps area. Long story short: mediawiki.org, likely in public, though we can anonymize prior to publishing any quotes, etc.

We'll also need to define how the privacy statement is presented to participants (I'll be discussing this with Aeryn via email)

I'd suggest you do this via a google form and send it to prospective participants via email.

On a side note, would it be possible to have unsubscribes from the recruitment survey be handled by OTRS or another email address?

Yes. We can have a mailto link you can put on wiki or in emails which would send unsubs to OTRS with a specific subject line.

JMinor awarded a token.