Page MenuHomePhabricator

Homepage: discovery of homepage after account creation (desktop)
Closed, ResolvedPublic

Description

The objective of this task is to help more newcomers discover their homepage. When it is built, it should be part of the "homepage treatment", meaning that those newcomers that are in the treatment group of the homepage experiment should see this addition, and the newcomers in the control group should not.

Potentially the most important time to help newcomers discover their homepage is the moment after account creation, which means after the welcome survey. The newcomer can do three things with the welcome survey: submit, skip, abandon. But no matter which of those paths from the welcome survey the newcomer takes, we still want them to discover their homepage, and we so may have to design different experiences for each path. So there are three paths from the survey:

  • Submit (~65%)
  • Skip (~12%)
  • Abandon (~22%)

Another important thing to keep in mind is that many newcomers may prefer to return to the context of their account creation, especially those who create their account while making an edit. They will especially want to go back and complete their edit so that they don't lose their work. These are three buckets of contexts that newcomers come from when creating their account:

  • Main page (~24%)
  • Reading (~57%)
  • Editing (~19%)

And then there are a set of things we could do to draw the newcomer's attention to their homepage after account creation include (but are not limited to):

  • Redirecting them straight to the page.
  • Adding a guided tour popup pointing to the personal tools link. Here is a demo tour.
  • Some other kind of affordance for the personal tools link.
  • Other things we may think of.

We decided to mostly rely on the GuidedTour tool (for users with Js activated), and laid out a matrix to map the treatment newcomers will receive according to what kind of actions they have taken during the survey and from which context they were coming prior to start the account creation process.

GuidedTour contents

There will only be two different GuidedTour popups, they will share the same layout, but they will differ in their content. Below are specifications for the content of the popups, but we have a question for the engineers: when working with the GuidedTour code, we would appreciate if you notice any additional flexibility or ideas for the popups. Can they include images? Colors? Different styling? Let us know. But these are the specifications:

  • When on homepage
    • Header: "Welcome to your homepage!"
    • Text: "You can always click on your username to return here."
    • Button: "Got it"
  • When not on homepage
    • Header: "Get started here!"
    • Text: "Click on your username to visit your homepage."
    • Button: "Got it"

Instrumentation

We will want our instrumentation to include visits to the homepage generated by these features. For users who come to their homepage from the survey confirmation screen button, we will want to record that they came from that button. For users who come to their homepage after seeing the GuidedTour, we also want to record that. Would it be possible to record the source as from the GuidedTour if the user clicks their personal tools link while the GuidedTour is open? Or maybe immediately after closing the GuidedTour? Looking for recommends from the engineers here.

Outline

Below is the outline showing the context of origin, the action taken by the user and the destination screen with the relevant GuidedTour treatment. The mockups will help with UI layout, positioning and wording.

  • Submits welcome survey
    • From main page: survey confirmation screen's only option is to go to homepage Fig. A1. Homepage shows GuidedTour to point at personal tools link, explaining how to return to homepage Fig. B1. Previous copy under the "Getting Started" header that contains tutorial and help desk links is removed.
    • From anywhere else: survey confirmation screen gives two options, one for homepage and one for original context Fig. A2. Previous copy under the "Getting Started" header that contains tutorial and help desk links is removed.
      • When choosing original context, redirect straight to original context with GuidedTour to point at personal tools link to invite discovering the homepage after reading Fig. B2.
      • When choosing homepage, the homepage shows GuidedTour to point at personal tools link, explaining how to return to homepage Fig. B1.
  • Never receives survey / Skips welcome survey (clicks the button on survey to skip)
    • From main page: redirect straight to homepage with GuidedTour to point at personal tools link, explaining how to return to homepage Fig. B1.
    • From anywhere else: redirect straight to original context with GuidedTour to point at personal tools link to invite discovering the homepage Fig. B2.
  • Doesn't click buttons on the survey confirmation screen / Abandons welcome survey (navigates away from survey with clicking either button)
    • To whatever page: a GuidedTour points at personal tools link to invite discovering the homepage Fig. B4.

Fig. A1


Fig. A2

Fig. B1

Fig. B2

Fig. B4

Note

Only users who create their accounts after this feature is deployed should see the GuidedTour. It should not appear for users who previously created their accounts.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Catrope renamed this task from Homepage: discovery of main page after account creation to Homepage: discovery of homepage after account creation.May 9 2019, 8:26 PM

@MMiller_WMF I have spent some time thinking about this and expanded the matrix below. Happy to hear your thinking before creating a click-through prototype to test the various scenarios.

  • Submits welcome survey
    • From main page: survey confirmation screen's only option is to go to homepage. Homepage shows GuidedTour to point at personal tools link, explaining how to get back.
    • From reading: survey confirmation screen gives two options, one for homepage and one for original context.
      • When choosing homepage, the homepage shows GuidedTour to point at personal tools link, explaining how to get back.
      • When choosing original context, redirect straight to original reading context with GuidedTour to point at personal tools link to invite discovering the homepage.
    • From editing: survey confirmation screen gives two options, one for homepage and one for original context.*
      • When choosing homepage, the homepage shows GuidedTour to point at personal tools link, explaining how to get back.
      • When choosing original context, redirect straight to original editing context with GuidedTour to point at personal tools link to invite discovering the homepage after editing.
  • Skips welcome survey (clicks the button on survey to skip)
    • From main page: redirect straight to homepage with GuidedTour to point at personal tools link, explaining how to get back.**
    • From reading: redirect straight to original reading context with GuidedTour to point at personal tools link to invite discovering the homepage.
    • From editing: redirect straight to original editing context with GuidedTour to point at personal tools link to invite discovering the homepage after editing.

I'm feeling like Abandons welcome survey by clicking anywhere on the chrome and never receive survey are two different scenarios.
Never receiving the welcome survey after account creation would potentially lead to same paths one gets when skipping the welcome survey:

  • Never receive survey
    • From main page: redirect straight to homepage with GuidedTour to point at personal tools link, explaining how to get back.**
    • From reading: redirect straight to original reading context with GuidedTour to point at personal tools link to invite discovering the homepage.
    • From editing: redirect straight to original editing context with GuidedTour to point at personal tools link to invite discovering the homepage after editing.

When a user Abandons the welcome survey instead, I would focus on where they are going rather than where they are coming from:

  • Abandons welcome survey (navigates away from survey with clicking either button)
    • To main page: a GuidedTour points at personal tools link or a banner is placed in the central notice space to invite discovering the homepage.
    • To anywhere else: a GuidedTour points at personal tools link to invite discovering the homepage.

Open questions:
* When a user creates a new account from the editing context and then decides not to go back to the original editing context will they lose the contribution draft?
** When a user skips or abandons the welcome survey, do we want - at second login - to send them an echo notification and/or show a banner on top of the homepage to remind them to fill out the survey?

Cntlsn reassigned this task from Cntlsn to MMiller_WMF.May 27 2019, 3:59 PM

@Cntlsn -- thanks for filling in the matrix. I see your plan makes heavy use of GuidedTour, but that's only one of the ways we could approach this. Do you think GuidedTour is best?

To answer some of your questions:

  • If a user creates a new account from the editing context and then decides not to go back there, they will lose their work. How should we account for that?
  • If a users skips or abandons the welcome survey, it would be great if they could take it again, but we haven't pursued that yet. Partially because we're not using the welcome survey responses to personalize the homepage yet. It is ticketed in T212021.

I think it is correct to separate abandoning the survey from not receiving the survey. But on that point, you say that if someone abandons to the main page, they should either get the GuidedTour or a main page banner. But in T222847, I think we're considering giving such a banner (or whatever it turns out to be) to all users in treatment groups, and leaving it there as long as they are logged in. How are you thinking about this?

This also made me realize that there is a way to escape finding out about the homepage. You could take the survey and then not click either of the buttons on the confirmation screen -- i.e. you could abandon the confirmation screen. What should happen to those users?

Finally, how should we approach all this for mobile? Should we address that in a separate task?

Cntlsn added a comment.EditedJun 3 2019, 2:15 PM

@MMiller_WMF

I see your plan makes heavy use of GuidedTour, but that's only one of the ways we could approach this. Do you think GuidedTour is best?

I think that if our goal is to "teach" and train users how to get to (or get back to) their homepage, then the guided tour popup is the best tool we have right now to indicate users the exact place (link) they have to click to reach the Homepage.

To answer some of your questions:

  • If a user creates a new account from the editing context and then decides not to go back there, they will lose their work. How should we account for that?

I think the button to go to the Homepage should feature some text below that states something like "by clicking this button you will leave the editing context, hence you will lose all your unsaved changes"

  • If a users skips or abandons the welcome survey, it would be great if they could take it again, but we haven't pursued that yet. Partially because we're not using the welcome survey responses to personalize the homepage yet. It is ticketed in T212021.

I missed that task, sounds great!

I think it is correct to separate abandoning the survey from not receiving the survey. But on that point, you say that if someone abandons to the main page, they should either get the GuidedTour or a main page banner. But in T222847, I think we're considering giving such a banner (or whatever it turns out to be) to all users in treatment groups, and leaving it there as long as they are logged in. How are you thinking about this?

The reason why I didn't go into further details is that I think this is something we need to discuss asap, preferably live. I'm feeling like the guided tour and the banner are serving two slightly different purposes here.

This also made me realize that there is a way to escape finding out about the homepage. You could take the survey and then not click either of the buttons on the confirmation screen -- i.e. you could abandon the confirmation screen. What should happen to those users?

I think this scenario would also fall into the Abandons welcome survey, or basically just follow the same approach: we should treat this differently according to where the user abandons the survey confirmation screen to.

Finally, how should we approach all this for mobile? Should we address that in a separate task?

Yes, mobile will have a different treatment as guided tour popups might be tricky to support on different mobile screen sizes and browsers. I created a new task here T224883

MMiller_WMF added a comment.EditedJun 6 2019, 6:51 PM

@Cntlsn -- thanks for responses. I think we've decided and learned a couple things this week allowing us to move forward:

  • Once users have left the editing context to create their account, any editing work will already have been lost (or if using VE, potentially lost). Continuing on to the homepage after account creation does not cause them to lose it or increase the risk that they will lose it, so we don't need to message around that.
  • We've decided to hold on the "addition to main page" feature (T222847), so we don't have to worry about redundancy or conflict with that.

Given all of this, I think we're just going to need two GuidedTour texts, one for when you're looking at the homepage and one for when you're not:

  • When on homepage
    • Header: "Your homepage"
    • Text: "Click your username to return here."
    • Button: "Got it"
  • When not on homepage
    • Header: "Get started"
    • Text: "Click your username to visit your homepage."
    • Button: "Got it"

So now, @Cntlsn, I think you have what you need to finalize the matrix and to make any necessary mockups for the survey confirmation screen and GuidedTours. And please let me know if you think the text of the GuidedTour should be different. It may be that if you look into GuidedTours you see that we have different options for the header and buttons, which we may not need.

Also, GuidedTours as JS-only. Do you see an easy thing to do for no-JS users?

Cntlsn updated the task description. (Show Details)Jun 7 2019, 3:26 PM
Cntlsn added a comment.Jun 7 2019, 3:59 PM

@MMiller_WMF
Thanks for your comment and for making some clarity on the state of the task.

I have updated the task description with the updated matrix and mockups relevant to the various states and placement of survey confirmation screen buttons and GuidedTour popups.

I have edited the language of the GuidedTour popups to make it a bit more conversational, since we're welcoming and inviting our users to discover the Homepage, and to match the feedback received during test result (users looking for a "start here" / "get started here" cue):

  • When on homepage
    • Header: "Welcome to your homepage!"
    • Text: "You can always click on your username to return here."
    • Button: "Got it"
  • When not on homepage
    • Header: "Get started here!"
    • Text: "Click on your username to visit your homepage."
    • Button: "Got it"

Feel free to change it if you're feeling it doesn't match the wiki tone of voice.


About the treatment for no-Js users, I was planning to show them banners in the sitenotice area (and a modal for when they go back to editing context), but I did some research and apparently none of those are supported in no-Js wiki.
So my idea would be to rely on echo notifications in this scenario. Whatever the context of origin and the destination, I would suggest to slightly change the current "welcome" echo notification to say:

  • Header: "Welcome to Wikipedia, [username]! We're glad you're here."
  • Body: "Looking for a way to get started? Click on your username to visit your homepage."

Here is a mockup of how it would look like on the no-Js notifications page

Potentially we could actually change this notification also for the Js-enabled users.

Happy to hear your thoughts about it!
And in case I'm missing something about the no-Js possibilities I'll be happy to review my proposal.

MMiller_WMF updated the task description. (Show Details)Jun 7 2019, 5:37 PM
MMiller_WMF updated the task description. (Show Details)

This is now ready for development.

I simplified the specifications in the description because there won't be a difference between whether the user comes from the reading or editing context. The welcome survey confirmation says the same thing either way, and so will the GuidedTour.

For no-JS users, I split that out into a separate task: T225318.

MMiller_WMF removed Cntlsn as the assignee of this task.Jun 7 2019, 6:00 PM
MMiller_WMF updated the task description. (Show Details)
MMiller_WMF updated the task description. (Show Details)Jun 11 2019, 3:08 PM
MMiller_WMF updated the task description. (Show Details)
MMiller_WMF renamed this task from Homepage: discovery of homepage after account creation to Homepage: discovery of homepage after account creation (desktop).Jun 12 2019, 4:37 PM

when working with the GuidedTour code, we would appreciate if you notice any additional flexibility or ideas for the popups. Can they include images? Colors? Different styling? Let us know.

@Cntlsn @MMiller_WMF sure, I think we could include images and implement custom styling.

For users who come to their homepage from the survey confirmation screen button, we will want to record that they came from that button.

OK. I'll probably add referer_route: welcomesurvey to the HomepageVisit schema.

For users who come to their homepage after seeing the GuidedTour, we also want to record that. Would it be possible to record the source as from the GuidedTour if the user clicks their personal tools link while the GuidedTour is open? Or maybe immediately after closing the GuidedTour? Looking for recommends from the engineers here.

What would "immediately after" be defined as, 5 seconds? I think we could have the guided tour change the query parameter to "fromguidedtour" so when the user lands on Special:Homepage, we could record the referer_route: guidedtour in HomepageVisit. But maybe it makes more sense to shoehorn this value into referer_action (current values are: view, edit, other), so we'd have referer_action: guidedtour. @nettrom_WMF thoughts on what you would prefer here?

when working with the GuidedTour code, we would appreciate if you notice any additional flexibility or ideas for the popups. Can they include images? Colors? Different styling? Let us know.

@Cntlsn @MMiller_WMF sure, I think we could include images and implement custom styling.

Cool, thanks! I have design options that include illustrations, and we might prefer those to the text-only dialog.
But, if you are already working on this, we should maybe release it with the current specifications, and then enhance the style later. How does that sound?

Cool, thanks! I have design options that include illustrations, and we might prefer those to the text-only dialog.
But, if you are already working on this, we should maybe release it with the current specifications, and then enhance the style later. How does that sound?

I think releasing incrementally is a good idea. Let's create a follow-up task for that please.

For users who come to their homepage from the survey confirmation screen button, we will want to record that they came from that button.

OK. I'll probably add referer_route: welcomesurvey to the HomepageVisit schema.

That sounds great to me!

For users who come to their homepage after seeing the GuidedTour, we also want to record that. Would it be possible to record the source as from the GuidedTour if the user clicks their personal tools link while the GuidedTour is open? Or maybe immediately after closing the GuidedTour? Looking for recommends from the engineers here.

What would "immediately after" be defined as, 5 seconds? I think we could have the guided tour change the query parameter to "fromguidedtour" so when the user lands on Special:Homepage, we could record the referer_route: guidedtour in HomepageVisit. But maybe it makes more sense to shoehorn this value into referer_action (current values are: view, edit, other), so we'd have referer_action: guidedtour. @nettrom_WMF thoughts on what you would prefer here?

There's six Guided Tour-related schemas defined, and from a glance at the Data Lake it looks like there's still data being logged there. If we're interested in figuring out if the user clicked through to the Homepage within some amount of time after dismissing the Guided Tour (or while still having it open), we should be able to combine the schemas to determine that. While that does mean we're shifting the burden of determining this from software to data analysis, I think that's a better option than coding for this specific case. @kostajh does that sound good to you?

@MMiller_WMF : I'd like to flag that I'd like to know more about measurement needs when it comes to the various discovery-related tasks that we now have going. It might be that all we do is put together a notebook that can run and keep us updated, but I'd like to be able to plan for that. Let me know what thoughts you have around this, and whether there's a scheduled time for deployment that I should be targeting. Maybe also get a phab task up for that.

Change 521545 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Homepage: discovery of homepage after account creation

https://gerrit.wikimedia.org/r/521545

While that does mean we're shifting the burden of determining this from software to data analysis, I think that's a better option than coding for this specific case. @kostajh does that sound good to you?

Works for me! If you change your mind and want this information encoded as an action parameter let me know.

@MMiller_WMF @Cntlsn just to clarify, we are getting rid of the "Getting started with editing" subheader, paragraph, and two links below, correct?

@kostajh -- yes. It was in the mockups, but not in the specifications. I'll add it now.

MMiller_WMF updated the task description. (Show Details)Jul 9 2019, 11:25 PM

@MMiller_WMF @Cntlsn the mock up says "Close and go back to article" but in production right now we have "Close and go back to $1" where $1 is the page that the user was on when they registered (not necessarily an article; it could have been a Special page). I assume we want to keep "Close and go back to $1" but let me know please.

@kostajh sure, even better of course!

My only question would be: is it possible that we send the user back to a page where we can't display the discovery aid?

My only question would be: is it possible that we send the user back to a page where we can't display the discovery aid?

Not if I have understood the specifications in this task. The only place the discovery aid does not load in my patch is on Special:WelcomeSurvey and Special:Homepage (the welcome tour loads on the latter).

It looks like the question is answered -- but this all sounds good to me!

Change 521545 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Homepage: discovery of homepage after account creation

https://gerrit.wikimedia.org/r/521545

@kostajh -- I just opened up Beta Wiki using an account that's existed and had the homepage turned on for months, and I received the GuidedTour. Does that mean, as currently built, all users who currently have their homepage on will receive a GuidedTour when the feature goes to production? I think we would rather not do that -- is there some way?

@MMiller_WMF we could write a maintenance script that sets the "I've seen this tour" hidden preference for all existing users in our experiment groups in kowiki, cswiki, and viwiki.

Alternatively we could have a patch to the tour loading code which checks for account age, so if your account is older than X number of days, we consider you to have seen the tour. @Catrope @Tgr do you have thoughts on this?

Tgr added a comment.Jul 17 2019, 9:43 PM

You could probably set $wgDefaultUserOptions so "seen the tour" is the default and the other needs to be set explicitly in the DB, and do that on registration. Although as long as it's experimental and only done on a few medium-sized wikis it might be easier to just set the flag via an SQL query.

Some unrelated thoughts:

  • how does this interact with editing? if the tour is shown while the user is editing, the user might assume that they can click the link and will be able to find their way back to the editing context afterwards.
  • will new user messaging from other products interfere? on some wikis, when you start editing you get several dialogs from VE and at least one from CX, if the homepage tour is shown on the same pageview that's not a great user experience. (Also the CX is one is shown in the same location, under the user toolbar.)
  • is it a good idea to always show such messages, even in contexts which are neither reading nor editing? E.g. when someone without a wiki account uses an OAuth-based external tool, they get sent to Wikipedia, where they get a login screen and after a successful login the OAuth authorization dialog, then get sent back to the tool. (The WikiEdu dashboard relies on this heavily IIRC.) Showing guidance about the home page (or a survey for that matter) in the middle of that flow would be pretty distracting.

@kostajh -- I am going to add to the specifications in the description that users who have already created their account before this feature should not receive the GuidedTour.

@Tgr -- thank you for thinking hard about this and bringing up these points.

  • About interacting with editing: yes, as currently specified, if the user has created their account from the editing context and then returns to that context, they'll see the GuidedTour there. I see what you mean: the call to action could make users think that if they click, they won't lose their work on the edit so far. I'm unsure whether this is an issue we need to worry a lot about. The reason is because there are a lot of blue links around the page which all have the same potential problem. And if a user does click mid-edit, I think they usually get a warning from their browser that says "Are you sure?" Is there some other behavior you think could be good?
  • About other new user messaging: this is something we have worried about in the past. We worried about it here with the welcome survey, and we have also talked about it here because English Wikipedia already has multiple clashing popups. I guess the challenge for me here is that it's hard to know which popups are active on which wikis under which conditions. Do you know a good way to figure that out so we can tell whether we'll have issues in our target wikis (Czech, Korean, Vietnamese, Arabic...and later Hungarian, Basque)?
  • About OAuth: I hadn't thought about that at all. So what would the experience be like? The brief moment when someone is on wiki to click the OAuth button, the popup would be there? Is there something we can do about that?
MMiller_WMF updated the task description. (Show Details)Jul 18 2019, 12:12 AM

Change 524682 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Only show Guided Tours to newly created user accounts

https://gerrit.wikimedia.org/r/524682

You could probably set $wgDefaultUserOptions so "seen the tour" is the default and the other needs to be set explicitly in the DB, and do that on registration. Although as long as it's experimental and only done on a few medium-sized wikis it might be easier to just set the flag via an SQL query.

I went with this approach, and created T228602 for specifically this issue.

Change 524682 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Only show Guided Tours to newly created user accounts

https://gerrit.wikimedia.org/r/524682

Etonkovidova added a comment.EditedJul 23 2019, 10:26 PM

@kostajh -- I am going to add to the specifications in the description that users who have already created their account before this feature should not receive the GuidedTour.

  • About other new user messaging: this is something we have worried about in the past. We worried about it here with the welcome survey, and we have also talked about it here because English Wikipedia already has multiple clashing popups. I guess the challenge for me here is that it's hard to know which popups are active on which wikis under which conditions. Do you know a good way to figure that out so we can tell whether we'll have issues in our target wikis (Czech, Korean, Vietnamese, Arabic...and later Hungarian, Basque)?

@MMiller_WMF to find out which extensions are installed where - you may take a look at InitialiseSettings.php. The most obvious extension that interfere is GettingStarted which installed on kowiki and viwiki. The other guided tours that potentially might interfere with homepage guided tour are somewhat more challenging to track down (prefix:MediaWiki:Guidedtour-tour can be used on wikis to see what tours they have).

Some examples of overlapping guided tours (and GettingStarted extension):

@nettrom_WMF The Homepage schema successfully records coming from welcomesurvey:

+----------------------+-------+
| event_referer_route  | count |
+----------------------+-------+
| personaltoolslink    |   168 |
| specialcontributions |    14 |
| specialconfirmemail  |    14 |
| userpagetab          |    12 |
| direct               |     5 |
| specialwelcomesurvey |     5 |
| other                |     4 |
| usertalkpagetab      |     2 |
+----------------------+-------+

Some examples of overlapping guided tours (and GettingStarted extension):

@Etonkovidova Is this something likely to happen, or an extreme scenario?
Could it maybe be a quick fix to set an order of appearance of the popups, to avoid displaying them all at the same time?

@Etonkovidova @kostajh @Catrope -- I just opened up tabs in production arwiki and production cswiki, and I received the "Get started here" popup (even though my accounts were created a long time ago). Did our effort to refrain from showing those to users who already had the homepage not work?

I’m not sure we SWATed the patch to prevent this.

I’m not sure we SWATed the patch to prevent this.

I'm pretty sure it caught the wmf.15 train, though? Which patch are you referring to?

It doesn't seem to me like there's anything to SWAT:

$ git log origin/wmf/1.34.0-wmf.15..origin/master --oneline
7017917 (origin/master, origin/HEAD) Localisation updates from https://translatewiki.net.
943ce85 (cog-popup-alignment) Help panel: Fix cog popup alignment

I'm pretty sure it caught the wmf.15 train, though? Which patch are you referring to?

Sorry, was going by memory. Your patch should fix the problem.

MMiller_WMF closed this task as Resolved.Jul 30 2019, 1:15 AM

I just tried this in production. It is working well. Thank you!