Page MenuHomePhabricator

Homepage: allow testing the homepage for different users
Closed, ResolvedPublic

Description

Feature: Ability to see how Special:Homepage renders for different users.

Since the homepage is private, we can't have undocumented URL parameters like we often use to test features before they become public. Any on-wiki way to do this would require at least a new right assigned to a restricted group.


Here are the things we would like to be able to do in order to test the homepage.

Most of the modules on the homepage contain information that are specific to each user. The impact module shows the pageviews around their edits, the mentorship module shows them the name of their assigned mentor, etc. We want to be able to make sure that the content shown for different users is appropriately tailored to them.

Therefore, it would be best if it were possible for us to see what the homepage would look like for any given user so that we can test a variety of users with different sorts of welcome survey responses and edit histories. A URL parameter would be good. I can also imagine a text box on the homepage where we can type a username.

One thing worth mentioning is that nothing we'll be showing on the homepage is secret or personally identifying.

Event Timeline

SBisson created this task.Feb 27 2019, 8:02 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 27 2019, 8:02 PM
MMiller_WMF renamed this task from Allow testing the homepage for different users to Homepage: allow testing the homepage for different users.Feb 27 2019, 8:03 PM
SBisson updated the task description. (Show Details)Feb 27 2019, 8:04 PM
MMiller_WMF updated the task description. (Show Details)Feb 27 2019, 11:47 PM
MMiller_WMF updated the task description. (Show Details)
MMiller_WMF edited projects, added Growth-Team (Current Sprint); removed Growth-Team.
kostajh added a comment.EditedMar 13 2019, 5:39 PM

Some thoughts from reading this:

A URL parameter would be good. I can also imagine a text box on the homepage where we can type a username.

Let's just do a URL parameter.

Any on-wiki way to do this would require at least a new right assigned to a restricted group.

What restricted group would this be?

One thing worth mentioning is that nothing we'll be showing on the homepage is secret or personally identifying.

The welcome survey responses are not public. Depending on what the account completion module looks like if the user has supplied, but not verified, their email, we would see what their email address is. Are both of those things OK?


Also mentioned on the meeting today, we want to be sure that the homepage is in read-only mode (account completion, mentorship, help modules), and that event logging is not occurring.


Last point, would it make sense to do this when we pick up the task recommendations work? I can see this being most relevant to verifying the content of the task recommendations module.

Now that we have specified exactly what will be in the initial version of the homepage, I think it is easier to provide some more detail here. The initial version will have the impact, account completion, help, and mentorship modules.

  1. Impact module: the most important thing we want to look at from the perspective of multiple users is the impact module. That's because we anticipate that users can have editing patterns that cause many edge cases in the module, and we want to see what those look like. Also, our test users won't be able to make many test edits in Korean, Czech, and Vietnamese Wikipedias in order to generate different configurations of the module.
  2. Account completion module: the second most important thing to look at. This is because different newcomers will have different calls to action depending on what activities they have already performed. However, we will be able to test this just by creating new accounts, since the calls to action do not include editing pages outside the User space.
  3. Mentorship module: the third most important thing to look at. We'll want to verify that different users are being assigned different mentors. We'll also want to see that questions asked to the mentors are listed in the module. We will also be able to test this by creating new accounts, because the mentors will know that we're testing when those accounts ask them questions.
  4. Help module: the fourth most important thing to look at. This module will look the same to all users until they ask a question, at which point we will want it to be listed with the module. Easy to check by creating new accounts.

So overall, in the near term, the one we want to figure out is the impact module. If that's the only one we need to do, does that make this an easier thing to achieve? Especially because that module definitely does not contain personal information or actions that the user can take?

That said, building a way just to see the impact module for different users postpones the larger challenge to another time. Regarding that prospect, here are some specifics I can think of:

  • I agree that we would not want to be able to make any edits on the user's behalf -- so the "read only" approach sounds right.
  • It would be best not to see the email address. Maybe such a page could say, "[email present]" in the text box instead of the actual email address.
  • I feel like the only people who would need to be able to use this functionality are the members of our team. But I don't have experience in the domain of user rights. Maybe @Catrope can weigh in.

Taking all that together, I've tried to describe all the specifications I can. I think the next step is suggestions from engineers on how we can best approach these needs.

kostajh claimed this task.Mar 19 2019, 3:32 PM
kostajh triaged this task as Normal priority.

Change 497841 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] [wip] Allow viewing impact module as rendered for other users

https://gerrit.wikimedia.org/r/497841

So overall, in the near term, the one we want to figure out is the impact module. If that's the only one we need to do, does that make this an easier thing to achieve? Especially because that module definitely does not contain personal information or actions that the user can take?

I've started working on an approach that just handles the impact module, for the reasons you described in T217281#5034705. A generalizable solution that would allow some set of privileged users to view Special:Homepage rendering for any other user is likely going to require a security review (a good thing!) and that will take some time.

In the meantime, the approach I'm using is to provide a special page (Special:Impact) that loads the impact module for the current user, or, with a URL parameter (?username=Foo), the username provided via the URL. @MMiller_WMF As a bonus, I've made this page transcludable so on any wikitext page you could add {{Special:Impact|username=Foo}} but I can remove that if you think that's undesirable.

Thanks, @kostajh -- so this page would be accessible (though not easily discoverable) to any user? Who could view the impact module for any other user?

so this page would be accessible (though not easily discoverable) to any user?

Yes, although we could list it on Special:SpecialPages if we wanted to, and filter out any event logging data for users who are not part of our experiment.

Who could view the impact module for any other user?

With the WIP patch I have, any user could embed or view the impact module for any other user. Does that sound OK?

After discussing with the team today, we have decided to proceed with the special page approach. We do not want any events to be logged for these views.

I do not think we should enable the embedding capability until we have a design for how the module displays the username of the user whose data it is showing.

For the special page itself, I think it would be good if the page were to display a prominent note above the module saying which user's data it is, e.g. "Now showing data from User:MMiller (WMF)"

Change 497841 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Allow viewing impact module as rendered for other users

https://gerrit.wikimedia.org/r/497841

kostajh updated the task description. (Show Details)Mar 26 2019, 3:25 PM

Special:Impact is present in betalabs and testwiki - the info displayed is exactly the same data as on a user home page Impact module.

Examples:

Special:Impact seems to be working well to me. Leaving this in my column so we can check it on Czech, Korean, and Vietnamese.

MMiller_WMF closed this task as Resolved.May 30 2019, 12:16 AM

This is working well for me in production and I use it all the time. It's important to note that we may revisit this for future modules that are tailored to specific users.