Page MenuHomePhabricator

Encourage account creation
Closed, ResolvedPublic4 Estimated Story Points

Assigned To
Authored By
Charlotte
May 12 2020, 3:48 PM
Referenced Files
F31938229: log-in-flow.mp4
Jul 17 2020, 10:07 AM
F31918901: Screenshot_20200706-132137_Wikipedia Dev.jpg
Jul 6 2020, 8:26 PM
F31911069: 2020-06-29 19.40.39.png
Jun 29 2020, 6:37 PM
F31911074: 2020-06-29 20.34.07.png
Jun 29 2020, 6:37 PM
F31911061: 2020-06-29 20.27.52.png
Jun 29 2020, 6:37 PM
F31911064: 2020-06-29 19.40.39.png
Jun 29 2020, 6:37 PM
F31859967: sev5-01-ip-blocked.png
Jun 10 2020, 9:39 AM
F31846262: sev5-01-no-account-b.png
May 29 2020, 7:38 AM

Description

Why are we doing this?

The app has a number of features that appeal to logged-in folks, including Suggested edits. We want to devise calls to action to encourage users to create or sign into existing accounts.

Proposed designs

01020304
sev5-01-no-account-a.png (1×720 px, 168 KB)
sev5-01-no-account-b.png (1×720 px, 102 KB)
sev5-01-no-account-c.png (1×720 px, 152 KB)
sev5-01-log-in.png (1×720 px, 79 KB)

👉 Zeplin

Concept

Within Suggested edits
  • Suggested edits menu item is always shown.
  • Suggested edits snackbar (T231444) and pulse animation of the new menu item is visible for all users of the app.
  • Tapping the snackbar leads users to a screen that teases Suggested edits and encourages account creation (01).
    • Copy:
      • Title: Did you know that everyone can edit Wikipedia?
      • Description: Suggested edits is a new way to edit Wikipedia on Android. It helps you make small but vital contributions to Wikipedia. Our goal is to make editing easier and more accessible for everyone! Log in or join Wikipedia to get started.
  • Tapping LOG IN / JOIN WIKIPEDIA leads users to account creation rather than the log in page. This change is essential to encourage account creation (03).
  • Account creation and log in page have been visually refreshed. Input fields use styles introduced in T246897, please make sure they’re applied correctly (02, 03, 04).
  • After successfully logging, users will be redirected to Suggested edits home (see Zeplin).
In other places
  • Tapping LOG IN / JOIN WIKIPEDIA via contextual menu in Explore, Feed card on Explore or Reading lists teaser should lead users to account creation rather than the log in page as well.
  • ☝️ Same behavior as defined in “Within Suggested edits” above.

How do we measure success?

  • Track how many users are creating an account via Suggested edits menu item.
    • Track how many users start editing after newly creating an account via Suggested edits menu item (new acct w/ first SE-tagged edit within 48 hrs of acct creation event).
    • Track how many users keep editing after newly creating an account via Suggested edits menu item (7-day and 30-day).
  • Track how many users are logging in via Suggested edits menu item.
    • Track how many users start editing after logging in via Suggested edits menu item (login w/ first SE-tagged edit w/in 48 hrs of login event).
    • Track how many users keep editing after logging in via Suggested edits menu item (7-day and 30-day).
  • Also:
    • Measure before and after data (session length, app usage frequency for 30 days before/after acct creation event) of app users that are newly creating an account and then start editing.
    • Measure before and after data (session length, app usage frequency for 30 days before/after acct creation event) of app users that are logging in and then start editing.

Event Timeline

Charlotte renamed this task from Encourage account creation to [SPIKE] Encourage account creation.May 19 2020, 4:50 PM
Charlotte triaged this task as Medium priority.
Charlotte updated the task description. (Show Details)

There have two APIs that can check the IP to see if it is blocked, and they are checking on the application level in MediaWiki.
https://www.mediawiki.org/w/api.php?action=help&modules=query%2Bblocks for per-wiki blocks
and
https://www.mediawiki.org/w/api.php?action=help&modules=query%2Bglobalblocks for global blocks

There's also a possibility that the IP ranges got blocked on the traffic server level, we may need to have a chat with the traffic team before we start implementing it.

Good stuff - thanks @cooltey.

Now that we know we can tease account creation to only people who are likely to be able to create accounts (pending discussion with traffic folks), throwing this over to @schoenbaechler to figure out what the teasers should look like.

scblr updated the task description. (Show Details)

@Charlotte input in regards to copy ...

  • Tapping the snackbar leads users to a screen that teases Suggested edits and encourages account creation (01).
    • Copy:
      • Title: Did you know that everyone can edit Wikipedia?
      • Description: Suggested edits is a new way to edit Wikipedia on Android. It presents opportunities for small but vital contributions to Wikipedia. We would like to make contributing easier and more accessible for everyone. Log in or Join Wikipedia to get started.

... and in regards to the chapter How do we measure success is appreciated.

  • Track how many users are creating an account via Suggested edits menu item.
    • Track how many users start editing after newly creating an account via Suggested edits menu item.
    • Track how many users keep editing after newly creating an account via Suggested edits menu item.
  • Track how many users are logging in via Suggested edits menu item.
    • Track how many users start editing after logging in via Suggested edits menu item.
    • Track how many users keep editing after logging in via Suggested edits menu item.
  • Also:
    • Measure before and after data of app users that are newly creating an account and then start editing.
    • Measure before and after data of app users that are logging in and then start editing.

Also cc'ing @Dbrant and @SNowick_WMF for input in regards to the How do we measure success chapter in the task’s description.

Charlotte renamed this task from [SPIKE] Encourage account creation to Encourage account creation.May 28 2020, 1:32 PM
Charlotte updated the task description. (Show Details)

Thanks @schoenbaechler - I've tweaked the copy slightly. @SNowick_WMF, I've added some concrete suggestions of what we could/should measure to the task description. I am not sure that we can do a before/after comparison, but do let us know.

If this is for a new release we can do a before/after since we should have that data archived in mediawikihistory, I will just need to know date range for 'before' period.
I will need to find out if we can track "after logging in via Suggested edits menu item" - will find out and add to ticket T252550

@Charlotte @SNowick_WMF can you make sure we have a task for the work needed from Product Analytics so we can prioritize it? (It sounds like her work relates to "How do we measure success?"?)

Per our standup, I updated the screens here and on Zeplin with “Forgot password” ✅

Good stuff - thanks @cooltey.

Now that we know we can tease account creation to only people who are likely to be able to create accounts (pending discussion with traffic folks), throwing this over to @schoenbaechler to figure out what the teasers should look like.

@Charlotte and @schoenbaechler
Should we still need to call the API to see if the user's IP is blocked or not?
For what I've seen on the task description it does not mention checking the IP and the SE menu is always visible.

We should definitely still check it @cooltey. And if the IP is blocked, show this 👇

sev5-01-ip-blocked.png (1×720 px, 75 KB)

Zeplin

Tapping LOG IN / JOIN WIKIPEDIA via contextual menu in Explore, Feed card on Explore or Reading lists teaser should lead users to account creation rather than the log in page as well.

@schoenbaechler

There's some other places that lead to the Login page, should we change them to the same behavior as above?

There's some other places that lead to the Login page, should we change them to the same behavior as above?

@cooltey yes! 💥💥💥


CC @Charlotte just so you know → what we’re doing here is encouraging account creation par excellence. Analytics data is going to be crucial.

In T252550#6218314, @schoenbaechler wrote:

CC @Charlotte just so you know → what we’re doing here is encouraging account creation par excellence. Analytics data is going to be crucial.

Yes, have already talked through the tracking with @SNowick_WMF. Could you outline what you need @cooltey to do, Shay? Research questions are in the "measuring success" bit in the task description.

Hi @schoenbaechler ,

Suggested edits snackbar (T231444) and pulse animation of the new menu item is visible for all users of the app.

The current text is this: %s, did you know that everyone can edit Wikipedia?, and that's for logged in users. (%s will be replaced by username).

Can we replace the %s with Hi so that it makes sense for non-logged-in users?

Can we replace the %s with Hi so that it makes sense for non-logged-in users?

Nope nope nope, this would be a nightmare for localization. We should either not include the user's name (i.e. "Did you know that everyone can edit Wikipedia?"), or have a separate string for non-logged-in users.

Thanks, @Dbrant. I vote for the "Did you know that everyone can edit Wikipedia?", which is much simpler. (cc @schoenbaechler )

Thanks, @Dbrant. I vote for the "Did you know that everyone can edit Wikipedia?", which is much simpler. (cc @schoenbaechler )

Seems sensible to me.

Did you know that everyone can edit Wikipedia?

@cooltey do I understand correctly that you’re suggesting this change only for logged out users? We should definitely keep the username for logged in users as it converts better (a learning from hundreds of A/B tests from my time at Online Fundraising).

If yes, this sounds good to me!

@cooltey do I understand correctly that you’re suggesting this change only for logged out users? We should definitely keep the username for logged in users as it converts better (a learning from hundreds of A/B tests from my time at Online Fundraising).

Sure! It makes sense to me and I'll make an update to it.

Cooltey: for analytics, see Shay's comment on T253982

Cooltey: for analytics, see Shay's comment on T253982

Thanks @Charlotte, I will update the PR with the new source "suggestededits".

@cooltey, looks good so far!

The first two are already mentioned in T252041, but very important for this particular task:

01)

Prolong time of snackbar to 15 seconds, it’s currently disappearing to fast

02)

  • When logging in via Suggested edits:
      • Suppress “Turn on reading list sync” dialog
    • Suppress “Reading lists synced successfully” snackbar.
    • Suppress echo notifications.
      • Why? We want to take users to Suggested edits home without distractions.
    • Show Reading lists snackbar, echo notifications and dialogs when users go to a different area of the app.

03) Inconsistent disabled state of buttons. Can we use colorAccent with 50% opacity for it? See Zeplin

2020-06-29 20.27.52.png (2×1 px, 122 KB)
2020-06-29 19.40.39.png (2×1 px, 99 KB)
2020-06-29 20.34.07.png (2×1 px, 160 KB)

04) Also, it should not be possible to tap on this button when no credentials (username/pw) have been filled out. Isn’t there some kind of client validation for this

2020-06-29 19.40.39.png (2×1 px, 99 KB)

In T252550#6265292, @schoenbaechler wrote:

@cooltey, looks good so far!

The first two are already mentioned in T252041, but very important for this particular task:

01)

Prolong time of snackbar to 15 seconds, it’s currently disappearing to fast

02)

  • When logging in via Suggested edits:
      • Suppress “Turn on reading list sync” dialog
    • Suppress “Reading lists synced successfully” snackbar.
    • Suppress echo notifications.
      • Why? We want to take users to Suggested edits home without distractions.
    • Show Reading lists snackbar, echo notifications and dialogs when users go to a different area of the app.

03) Inconsistent disabled state of buttons. Can we use colorAccent with 50% opacity for it? See Zeplin

2020-06-29 20.27.52.png (2×1 px, 122 KB)
2020-06-29 19.40.39.png (2×1 px, 99 KB)
2020-06-29 20.34.07.png (2×1 px, 160 KB)

04) Also, it should not be possible to tap on this button when no credentials (username/pw) have been filled out. Isn’t there some kind of client validation for this

2020-06-29 19.40.39.png (2×1 px, 99 KB)

@schoenbaechler All done!

Looks good @cooltey — the only thing missing that’s defined in T252041 and relevant for this is:

  • Optimizations for “Hi %username, did you know that everyone can edit Wikipedia?” snackbar and its corresponding pulse animation:
    • Shown on the third time of using the app (for logged in and logged out users). A measure to avoid overlaps with messaging/notifications from other areas. It’ll give the snackbar more weight.

If that is fixed, ready for QA sign off!

In T252550#6276787, @schoenbaechler wrote:

Looks good @cooltey — the only thing missing that’s defined in T252041 and relevant for this is:

  • Optimizations for “Hi %username, did you know that everyone can edit Wikipedia?” snackbar and its corresponding pulse animation:
    • Shown on the third time of using the app (for logged in and logged out users). A measure to avoid overlaps with messaging/notifications from other areas. It’ll give the snackbar more weight.

If that is fixed, ready for QA sign off!

Hi @schoenbaechler
There's another snackbar that will show up. What would be the new logic of showing the snackbar below?

Screenshot_20200706-132137_Wikipedia Dev.jpg (2×1 px, 371 KB)

The current logic is:
If the "Hi %username, did you know that everyone can edit Wikipedia" snackbar has shown, then it will show up on the next time of using the app.

Hey @cooltey — thanks for checking! The initial idea of this snackbar was to introduce image tagging to users.

At this point, I think we should show this snackbar if users haven’t added any image tags via app yet, since we’re teasing the feature already with the “Did you know that everyone can edit Wikipedia” snackbar (T252041) and the editor retention notifications (T249629).

To be more precise: show the snackbar when none of the user’s contributions have been an image tag. It should be shown if users have made more than 10 edits.

I’m tempted to suggest removing this snackbar as we’re moving towards push notifications (T251439). But acknowledge that it doesn’t hurt to show it (until push notifications are a thing).

Hi @schoenbaechler

To be more precise: show the snackbar when none of the user’s contributions have been an image tag. It should be shown if users have made more than 10 edits.

We would not recommend to have this logic since it requires extra API calls before actually showing the snackbar. (total counts + image tags counts).

Would it be possible to just remove this snackbar?

Thx @cooltey — I’ll let @Charlotte have the last word on this, as removing the snackbar for image tags might lead less engagement. To summarize @Charlotte; either we keep this snackbar, like I described in T252550#6300608:

To be more precise: show the snackbar when none of the user’s contributions have been an image tag. It should be shown if users have made more than 10 edits.

I’m tempted to suggest removing this snackbar as we’re moving towards push notifications (T251439). But acknowledge that it doesn’t hurt to show it (until push notifications are a thing).

OR we remove it, as per @cooltey’s suggestion:

We would not recommend to have this logic since it requires extra API calls before actually showing the snackbar. (total counts + image tags counts).

Would it be possible to just remove this snackbar?

Let's remove it for now @schoenbaechler. We can go back to the drawing board about engagement.

Cheers @cooltey for letting us know.

Thx @cooltey — I noticed a small detail after going through the flow as a first time user.

There’s a redirection to the initial state of the screen (“Did you know that everyone can edit Wikipedia?”) after logging in (00:05 in the video) which is not ideal. We need to lead users directly to Suggested edits home after the login without this intermediary step. Could you look into optimizing this?

In T252550#6314770, @schoenbaechler wrote:

Thx @cooltey — I noticed a small detail after going through the flow as a first time user.

There’s a redirection to the initial state of the screen (“Did you know that everyone can edit Wikipedia?”) after logging in (00:05 in the video) which is not ideal. We need to lead users directly to Suggested edits home after the login without this intermediary step. Could you look into optimizing this?

Of course, will do!