Page MenuHomePhabricator

Create link to in-app user feedback collection on Mediawiki (survey dialogue)
Closed, ResolvedPublic

Description

Background

As we work on enhancements to the messaging interface in the Android app, we want to gain an understanding of gaps in our users workflows. One of our methods for collecting this information will be an in-app survey.

The Task

This task is to develop a dialog for the survey we will deploy with our target audiences.

Requirements

Narrowed to target demos with required translations:

  • English Wikipedia users/editors in India and Nigeria
  • Arabic and French Wikipedia users/editors from Morocco, Egypt, DR Congo and Mali
  • Hindi Wikipedia users/editors
  • Indonesian Wikipedia users/editors
  • Japanese Wikipedia

Other:

  • Able to display text and images
  • Receives sign off from designer and analyst
  • Hard code in snackbar and display after a user submits a post on a talk page (user or article)

Format

  • Post on MediaWiki, similar to the usability protocol in July 2021
  • Use custom tab activity to display contents in-app (T291969)

Dialog

Links

Per a previous Slack conversation with @Dbrant:

rs: can we implement a variable in the dialog link that points directly to the corresponding language? e.g. when users have set their primary language to Arabic can we take them to the Arabic version of the protocol?
db: Yep

Bahasa Indonesia (id):
https://forms.gle/8EgVWERw4FnqGSVv7

English (en):
https://forms.gle/8W2kYaRgADEL6cTG8
→ also use 'en' as the fallback

French (fr):
https://forms.gle/WxRP7NSUttyQUQgs7

Arabic (ar):
https://forms.gle/TtVov1y2ggkSU5jz6

Japanese (ja):
https://forms.gle/NYEmn2PWoxicGEh87

Hindi (hi):
https://forms.gle/PzechN4hi5rar7a18

Dialog

Title:
Help us improve Talk Pages

Copy:
Please help us improve your talk page experience by taking a brief survey.

CTA 1:
TAKE SURVEY

CTA 2:
READ PRIVACY POLICY

CTA 3: cancel

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
JTannerWMF renamed this task from Create in-app survey infrastucture for upcoming user testing to Create link to in-app survey for upcoming user testing.Sep 27 2021, 4:13 PM
JTannerWMF updated the task description. (Show Details)
LGoto renamed this task from Create link to in-app survey for upcoming user testing to Create link to in-app user feedback collection on Mediawiki.Oct 5 2021, 4:28 PM
LGoto assigned this task to scblr.
LGoto added a subscriber: Johan.

@schoenbaechler

Hard code in snackbar and display after a user submits a post on a talk page (user or article)

After a user submits a post on a talk page, we already show a snackbar that allows the user to "undo" the message. When can we show the survey snackbar so that it doesn't interfere with the Undo snackbar?

Per our discussion in a standup last week, we’re going to show the survey after the undo snackbar (sequenced).

Per our discussion @Dbrant — I added the snackbar copy to the task’s description!

Per our discussion @Dbrant — I added the snackbar copy to the task’s description

@schoenbaechler You may test out the snackbar and the destination page by using this build:
https://github.com/wikimedia/apps-android-wikipedia/actions/runs/1338220227

To test it, go to Developer settings, scroll all the way to the bottom, and tap "Show talk page survey".

@Dbrant

Just tested the links in all languages — worked fine!

Question re: translations — did you add the snackbar text to Translatewiki.net yet? I think the sooner the better. Thanks!

@schoenbaechler The strings are on TranslateWiki as of now:
https://translatewiki.net/wiki/Wikimedia:Wikipedia-android-strings-talk_snackbar_survey_text
https://translatewiki.net/wiki/Wikimedia:Wikipedia-android-strings-talk_snackbar_survey_action_text

We might need to get them translated pro-actively if we want them to be completed by next week.

FYI @Dbrant I’m receiving translations for these here:

https://docs.google.com/document/d/1Tf3FkYVmVTtJxdVUkmXgC2ZIAkcgH2AFaF4adMQrL5E/edit

We have all except for Hindi yet — maybe @Sharvaniharan can help out with the Hindi translation and add it to the Gdoc?

The Hindi translation just arrived!

Thanks!

@Dbrant — could you add the translations to translatewiki and release it? 👨‍🚀 Thanks a lot! 🚀

scblr renamed this task from Create link to in-app user feedback collection on Mediawiki to Create link to in-app user feedback collection on Mediawiki (survey snackbar).Oct 19 2021, 7:52 AM

So this has to be a dialog instead of a snackbar due to constraints from the Legal department. I've updated the task. I have dropped in the link for English, feel free to start there, I will have the translated links in tomorrow before 9AM ET

LGoto renamed this task from Create link to in-app user feedback collection on Mediawiki (survey snackbar) to Create link to in-app user feedback collection on Mediawiki (survey dialogue).Oct 27 2021, 7:47 PM
JTannerWMF updated the task description. (Show Details)
JTannerWMF added a subscriber: scblr.
JTannerWMF updated the task description. (Show Details)

Is it possible for people to select maybe later and then get a reminder dialog the next time they open the app?

Is it possible for people to select maybe later and then get a reminder dialog the next time they open the app?

you mean the next time they post a Talk message?

hi @JTannerWMF

I have built this dialog like this [Seen here on explore screen, but will move it to the correct place shortly]:
First time:

first.png (2×1 px, 145 KB)

Second[Last] time:
last.png (2×1 px, 146 KB)

And the dialog is non-cancelable, meaning it won't go away if you click anywhere else on the screen. I felt it would prevent users from accidentally cancelling it. please let me know your thoughts.

@JTannerWMF @Dbrant
I am not sure if there is an easy way to host the form in a custom tab, but the custom tab by default seems to be opening the google form in an external browser. I suspect it is because of the authentication of the gmail account. Just wanted to flag this.
The privacy policy is shown in-app without issues.

Will update hi and ja after they are available.
If you want to test, it is available here: https://github.com/wikimedia/apps-android-wikipedia/pull/2948/checks. It is shown on launch for now.

Thank you so much for getting on this so fast @Sharvaniharan , yes the behavior @Dbrant shared is fine for the maybe later reminder. not being able to click out of it sounds good, also can we instrument the dialog (if it isn't instrumented already).

I'll have a look at the custom tab behavior and reflect a bit on it sending people outside of the app.

Another question, when someone clicks on the privacy policy, and want to go back, will it bring them back to this dialog?

Ok all of the languages are there *wipes brow*

hi @JTannerWMF

I have built this dialog like this [Seen here on explore screen, but will move it to the correct place shortly]:
First time:

first.png (2×1 px, 145 KB)

I thought we were going to put the link to the Privacy Policy inline with the text of the dialog? (Instead of it being a whole button)
That way the user would be able to go "back" to this dialog from reading the privacy policy.

@JTannerWMF @Dbrant
I am not sure if there is an easy way to host the form in a custom tab, but the custom tab by default seems to be opening the google form in an external browser. I suspect it is because of the authentication of the gmail account. Just wanted to flag this.

It looks like this is because the survey requires you to be logged into a Google account. (If I open it in an incognito window, it asks me to log into Google first)
@JTannerWMF I'm pretty sure you can configure the survey to not require being signed in to Google. Can we do that?

Give it a check now, I did it for all of the languages. If we can have the privacy policy inline to deprioritize it as a link, that would be wonderful!

Yep, that's much better. And regarding the links opening in an external browser instead of our Custom Tab, it looks like this happens when the "shortlinks" get redirected to the "full" url of the survey. If we hardcode the full URLs, instead of the shortlinks, they will open in a custom tab just fine. (@Sharvaniharan please take the full redirected versions of the survey links and hardcode those.)

@JTannerWMF @Dbrant Have incorporated all comments from here

  • Changed the button to link text for privacy policy
  • hardcoded the full urls, and now the customTab is working correctly. 👍
  • You can test it out with the updated version in the above link

@JTannerWMF @SNowick_WMF

What all do we want to instrument:

  1. Survey dialog shown.
  2. 'Take Survey' clicks
  3. 'Maybe later' Clicks
  4. Do we need the privacy policy clicks for anything?
  5. 'Cancel' clicks when it is shown the second time.

Can we just rely on the Google side to see how many times the URLs were opened?

JTannerWMF removed JTannerWMF as the assignee of this task.EditedNov 1 2021, 3:53 PM
This comment has been deleted.

Lets talk about this today. Because some people may see the snackbar but not actually click the link, they may abandon when looking at the privacy policy.

Title:
Help us improve Talk Pages

Copy:
Please help us improve your talk page experience by taking a brief survey.

CTA 1:
TAKE SURVEY

CTA 2:
READ PRIVACY POLICY

CTA 3: cancel

CTA 4: MAYBE LATER

@JTannerWMF ^^ If we want to roll out the survey as soon as possible, we will need to get these messages translated into the target languages, since we can't rely on TranslateWiki to get them done at any particular time. Right now we have translations for the snackbar as it was originally envisioned:

Thanks for using talk pages. Could you help us improve the app by participating in a survey?
GET STARTED

Do we want to reuse this text, instead of your updated dialog message? We would still need translations for the dialog title ("help us improve talk pages"), as well as "Read privacy policy" and "Maybe later".

@JTannerWMF As a compromise we could use existing translations for the message and CTAs, and remove the title of the dialog.
This is what it would look like, with no further translations necessary:

device-2021-11-01-152844.png (2×1 px, 217 KB)

How does that look?

That works, can you align the privacy policy bottom left?

  • show explicitly first time they go to a Talk page (article or user)
  • if they click "maybe later", then show it again after they publish a comment on a Talk page