Page MenuHomePhabricator

Remove mobile editing "call to registration" CTA
Closed, ResolvedPublic

Assigned To
Authored By
Nemo_bis
Jan 24 2015, 3:41 PM
Referenced Files
F56364: Screenshot_2015-03-05_16.25.54.png
Mar 6 2015, 12:35 AM
F44877: step_one.png
Feb 21 2015, 8:07 PM
F44879: step_two.png
Feb 21 2015, 8:07 PM
F44881: step_three.png
Feb 21 2015, 8:07 PM
F44882: step_four.png
Feb 21 2015, 8:07 PM
F42916: notify.png
Feb 18 2015, 10:02 AM

Description

When an unregistered user tries to edit a page in the mobile site, an interstitial is shown, which has a warning in yellow, a login button in blue, then in white "register" and finally "continue editing". Requested deliverables:

  1. "Continue editing" should be the first option, in blue.
  2. This warning should probably not be an interstitial, so that only one click is needed to reach the editing window. On devices which support JavaScript, it should be an alert which goes away by itself after some seconds.

Anecdotal evidence: a complaint by an unregistered user that mobile requires registration; an experienced user tries quickly and confirms; three other experienced users are unable to disprove the claim despite knowing of the recent changes in permissions (T85317). Clearly, we are conveying the incorrect message here.

Empirical evidence:

The data is not out here. We have lots of evidence that making the "continue editing" default or at least more prominent is desired.

login/signup should be more prominent.

We tried this on desktop and saw huge productivity losses. We were able to regain those some of those loses when we made "continue editing" clear and prominent.

See https://meta.wikimedia.org/wiki/Research:Asking_anonymous_editors_to_register

In my professional opinion (it is my job after all), driving anonymous editors to register does not seem to be meeting any of our goals -- to have more editors and to help those editors work productively.

Here, I think that personal opinion needs to bow to empirical observation.

Event Timeline

Nemo_bis assigned this task to Maryana.
Nemo_bis raised the priority of this task from to Medium.
Nemo_bis updated the task description. (Show Details)
Nemo_bis added subscribers: Maryana, Legoktm, Nemo_bis and 3 others.
gerritbot subscribed.

Change 186591 had a related patch set uploaded (by Florianschmidtwelzow):
Move "Edit without login" to the first position

https://gerrit.wikimedia.org/r/186591

Patch-For-Review

Hi @Nemo_bis, thanks for this task! I don't know, how Web-Team-Backlog team thinks about anonymous editing on mobile devices (sidenote: maybe you can review the edits from anonymous users on it wiki made with MobileFrontend in a specific time span and give a short feedback about the good/bad-rate?), the change of the order of the buttons is a comparatively easy change, so i uploded the change for it (see gerritbot).

Let's see, what mobile team (and especially Maryana) think about it, i want to give my +1 for it :)

sidenote: maybe you can review the edits from anonymous users on it wiki made with MobileFrontend in a specific time span and give a short feedback about the good/bad-rate?

I did not personally do this, but other users did and I posted the results at https://meta.wikimedia.org/wiki/It.m.wikipedia.org

FYI, really both login, create account, anon buttons should be progressive. They are both progressive actions.

I wonder if it would be better to drop sign up button and change login button to say "Edit with a username" and put a big OR piece of plain text in between. The second button would take you to login (which in term prompts you to create an account)

Let's keep experimenting with ideas and see what brings the most edits (I assume this is what we are optimising for).

Change 186591 merged by jenkins-bot:
Move "Edit without login" to the first position

https://gerrit.wikimedia.org/r/186591

Thanks. Point (2) still is to be implemented.

Did someone test (1) yet?

goes away by itself after some seconds

I'm not really comfortable with this, because of 2 things:

  1. it isn't the behavior on desktop (ok, that would be acceptable)
  2. we can not be sure, that the user reads the (entire) warning

So, maybe it should go away, if the user access the editor window (and don't select this input field by default, if the user isn't logged in) or when the user clicks on a "X" button inside the warning box.

Thanks. Point (2) still is to be implemented.

You're right. With your approval I'd like to break point 2 out into its own user story where we can discuss it without it being tied to point 1 so much.

Did someone test (1) yet?

Yes. I can also confirm that the "Edit without logging in" button is now progressive and is positioned above the "Log in" and "Sign up" buttons.

You're right. With your approval I'd like to break point 2 out into its own user story where we can discuss it without it being tied to point 1 so much.

Sure, it doesn't matter that much. However that would already be the third task created to address the finding that

driving anonymous editors to register does not seem to be meeting any of our goals

I think it would be relatively easy to move the cta to the end of the workflow (before the sites gets reloaded, e.g. a text like "Did you know, that you can login/register to Wikipedia?" and a button login/register and No thanks (which will reload the page). The bigger problem i have is: How we can solve the "You're not logged in, your ip will be saved..." problem aka "How we can make sure, that the message gets read by the user without to interrupt him tooo much? Especially on small screens :/

I think it would be relatively easy to move the cta to the end of the workflow (before the sites gets reloaded, e.g. a text like "Did you know, that you can login/register to Wikipedia?" and a button login/register and No thanks (which will reload the page).

Sounds nice. The re-addition in this form might be split to a separate task/follow-up changeset, if it's easier that way.

The bigger problem i have is: How we can solve the "You're not logged in, your ip will be saved..." problem aka "How we can make sure, that the message gets read by the user without to interrupt him tooo much? Especially on small screens :/

Does jquery.tipsy work in mobile browsers which support JavaScript? IIRC it's configurable to last at most a certain number of seconds *and* can be dismissed. The other semi-standard way to do this sort of stuff is mediawiki.notification.js. Both worth trying IMHO; the interstitial can be kept for the browsers we don't manage to support.

the interstitial can be kept for the browsers we don't manage to support.

No need, non JS browsers can't edit in mobile :)

Does jquery.tipsy work in mobile browsers which support JavaScript? IIRC it's configurable to last at most a certain number of seconds *and* can be dismissed. The other semi-standard way to do this sort of stuff is mediawiki.notification.js.

Hmm, i'll take a look :)

jQuery.tipsy isn't loaded on mobile actually, and i'm feeling bad to load the whole library just for one notification. Therefore i tested mediawiki.notify, because it's already targeted to desktop _and_ mobile. This would be the notification:

notify.png (371×392 px, 21 KB)

Actually, the message is stripped (i think there is some character limit, but i haven't checked it yet). I suggest an implementation without any autohide, so the user needs to confirm that he read the notice. The problem is, that the notification will normally hide the next button to save the edit, so in fact, it still will interrupt the workflow, but (in my personal point of view) much nicer and more gracefully as before. Opinions, maybe some Design input?

The screenshot looks rather good as a result. I'd still consider some conservative auto-hiding, like 10 seconds.

Ok, i played around and couldn't stop :P That's how my change (which i will upload soon) looks like:

step_one.png (658×373 px, 38 KB)
step_two.png (656×375 px, 31 KB)
step_three.png (658×378 px, 31 KB)
step_four.png (659×378 px, 36 KB)

I don't know, if step 4 is a good idea or if we want to kill the Cta completly with the change :/

Info:
A click on "No, thanks" simply reloads the page (like it does for logged-in users). The click on the other buttons redirects the user to the login/signup page.

Change 192095 had a related patch set uploaded (by Florianschmidtwelzow):
WIP: Move Editor Cta to the end of the editing process

https://gerrit.wikimedia.org/r/192095

Patch-For-Review

Hah, you have surpassed yourself. ;-) I like that you are also using the screen for the edit summary, which has "plenty" of space and where it's hopefully not much distracting. The final CTA is ok. It seems to me that your proposal achieves the goal with the least disruption over the current design.

There's always time to assess the efficacy of the final CTA and whether two anoneditwarnings are redundant.

Hah, you have surpassed yourself. ;-)

:))

whether two anoneditwarnings are redundant.

Yeah, actually they're redundant. Maybe we can remove the initial notification and use only the one in the editor preview (or vice versa) :)

Sure, it doesn't matter that much. However that would already be the third task created to address the finding that driving anonymous editors to register does not seem to be meeting any of our goals

Good point well made :)

@Florian: is there anything blocking you removing the WIP tag from your patch?

@Florian: is there anything blocking you removing the WIP tag from your patch?

@phuedx: in fact just the point, that i don't know, if two warnings are ok, but this can be handled "on the fly" here or in the change, so i can remove it :)

Have we done any analysis to check the effect of switching the edit anonymously button to the top? Have edits increased? Has quality been maintained?

It's worth pointing out that when we ran an A/B test to explore sign up / editing behaviour
In case A, when registering we took the user to a page with a blue pointer pointing them and encouraging them to edit http://en.m.wikipedia.beta.wmflabs.org/wiki/Headings?article_action=signup-edit
In case B we just threw them into the editor.

Case A got significantly more edits in the A/B test (https://gerrit.wikimedia.org/r/#/c/104790/ for some background)
So the interstitial may be helping us. Making the edit button throw the anonymous user straight into the editor might /lose/ us editors. I thus really think we should be data driven in anything regarding editing and I appreciate the data driven methods so far (e.g. analysing anon edit quality). Let's not guess this sort of thing and rely on anecdotal evidence or data from the desktop site (it's really like comparing apples to oranges).

From my personal point of view my main worry about moving this to the end is it will lead to lots of accidental anon edits. One of the things that frustrates me about desktop editing is I often edit whilst logged out and then expose my IP address (the interstitial on desktop is pretty hidden). If we move login / signup to the end it should also be possible to claim the edit. This could be achieved by doing an ajax login before the save which is no problem to do on https.

On subject of tipsy - no - please don't load this library. Every byte is precious. I'm not sure adding the warning over the header is a good idea from an accessibility point of view. I'm not quite sure what is needed from a legal point of view - for instance I'm guessing this could be moved to the edit summary screen if the goal is simply to throw them in the edit.

A few questions and a comment.

Can we technically attribute an edit post-save to a user who then logs in or make an account? If not we shouldn't imply that we can (or if the edit isn't actually saved yet we shouldn't imply that it has)

Have we reviewed the findings by the growth team on pre-and post call to action for account creation? @Halfak may be able to help here.

irrespective of when and where we have the call to action I think @Jdlrobson's suggestions come the closest

Login should be the primary action, create account should be the normal action and "Don't ask me to login" (we can wordsmith this) should be a, neutral, Quiet action.

Don't treat this as a design spec, but I imagine something like this being the idea.

Screenshot_2015-03-05_16.25.54.png (334×796 px, 26 KB)

Setting cookies for IP users is always a bit worrisome since "remembering" something frequently fails and leads to a worse experience.

I'd urge everyone who is interested in this work to hold tight until proper resources can be given to making sure that the influx of IP edits can be properly analyzed to make sure they are of a quality (and quantity) to justify the possible noise of bad edits.

It might even make sense to postpone until VE on (small screen) mobile is a reality, given the easy of accidentally making syntax issues from small screen edit sessions.

Change 192095 abandoned by Florianschmidtwelzow:
Move Editor Cta to the end of the editing process

Reason:
See discussion on bug report. This patch isn't lost, if here are some good starting points, it can be reactivated/revisited later or pieces can be used in new changes, too.

https://gerrit.wikimedia.org/r/192095

Can we technically attribute an edit post-save to a user who then logs in or make an account?

Not that i know a standard way (and it shouldn't be possible at all i think).

I'd urge everyone who is interested in this work to hold tight until proper resources can be given to making sure that the influx of IP edits can be properly analyzed to make sure they are of a quality (and quantity) to justify the possible noise of bad edits.

I think you're right, that needs more discussion.

Hey all. I have been keeping an eye on this task. I think that Nemo represents the conclusions of the Growth studies fairly. It seems that anything that we have tried to place between an anonymous editor and saving an edit resulted in dramatically decreased productivity (edits to articles that were not reverted) -- whereas the post-edit CTA, had insignificant impact on productivity. See https://meta.wikimedia.org/wiki/Research:Asking_anonymous_editors_to_register These results were consistent for all Wikis studied (English, German, Italian & French).

I'd advise against the common assumption that reducing the friction for anonymous editing will result in wave damaging edits. This is often speculated, but not once has it been supported with data. While I appreciate the call for a study, I think that we've reached a point where we should assume that opening up editing does *not* increase the proportion of damaging edits. I see the burden on those who claim it will cause damage to explain why we haven't seen evidence of this so far in the myriad of controlled and natural experiments.

In general, I'd advise strongly against making "register an account" a primary action and "continue anonymously" secondary. That strategy showed the largest losses in productivity in our experiments. Editing anonymously has historically been considered a legitimate activity and my experimental work suggests that was wise. At a glance, @Jaredzimmerman-WMF's mockup suggests that "create an account" and "login" are the only legitimate actions. It's only after careful reading that we see the grey text beneath the buttons is even clickable. It was during the view of a similar modal in the Growth experiments where we saw the most substantial drop-out. See the funnels from the study: https://meta.wikimedia.org/wiki/Research:Asking_anonymous_editors_to_register/Study_1#Funnels Even when we graduated "continue editing" to a button (still secondary), we still saw significant losses in productivity: https://meta.wikimedia.org/wiki/Research:Asking_anonymous_editors_to_register/Study_2 I understand that this UI fits with UX strategies, but it doesn't seem to work in practice.

Oh! I should also add that I'll be happy to perform a quality analysis of anon edits if we can set up a controlled study for this change.

Can we technically attribute an edit post-save to a user who then logs in or make an account?

Not that i know a standard way (and it shouldn't be possible at all i think).

We can use JavaScript API to create accounts/login though.. apps use it!
You could imagine adding this into the edit workflow and then saving only after registration / login - really doable.

Anonymous editing is primary action. Create a separate task for interstitial if you think that is still needed with details of the why we would do this and what we would expect it to achieve.