Page MenuHomePhabricator

Remove broken campaign=leftNavSignup code
Closed, ResolvedPublic

Description

A while back, back in 2013, we didn't have anonymous editing. When the left navigation menu was opened, we used to show a blue drawer inside the main menu prompting a user to sign up from the left nav.

We ran an A/B test where if an editor signed up to the mobile site, in this scenario, when returning to the page they were prompted by a blue pointer overlay to continue their editing journey.

Screen Shot 2018-09-24 at 12.27.42 PM.png (276×351 px, 28 KB)

The A/B test showed editors were more likely to edit when seeing this overlay, thus the code was shipped.

Fast forward to 2018, this code has been neglected and Bartosz has shown it to be broke in 2 fundamental ways:

  • There's no way to get into this treatment from the main menu (since there is no pointer overlay anymore)
  • When we do show it the url to start editing is broken!

We should thus remove this code.

Research notes which led to this code:

a research draft:
https://meta.wikimedia.org/wiki/Research:Mobile_editor_engagement/Left_navbar_signup_CTA_test

a schema draft:
https://meta.wikimedia.org/wiki/Schema:MobileLeftNavbarEditCTA

trello:
https://trello.com/c/nr63WBAG

Sign off steps

  • Given it was proved useful, do we want to relaunch this feature in any way? If so, please create a task and we can explore that.
  • Can we find the research related to this work?
  • Can we add any information to this task to document what this was, what we tested
  • and how it broke?

Event Timeline

Change 461848 had a related patch set uploaded (by Jdlrobson; owner: Bartosz Dziewoński):
[mediawiki/skins/MinervaNeue@master] Remove broken remains of &campaign=leftNavSignup handling

https://gerrit.wikimedia.org/r/461848

Jdlrobson updated the task description. (Show Details)

FYI @Tbayer, @Neil_P._Quinn_WMF and @ovasileva - We have a patch removing this code but I wanted to make sure you were aware of this and had a chance to catch up on the history of the research/motivations behind the feature (if you care to do so).

Skipping QA as removes dead and thus untestable code. Will let Olga take care of sign off steps.

Change 461848 merged by jenkins-bot:
[mediawiki/skins/MinervaNeue@master] Remove broken remains of &campaign=leftNavSignup handling

https://gerrit.wikimedia.org/r/461848

@MMiller_WMF you might be interested in this historical in-context help experiment.

Resolving as the work is done, but let's keep the conversation going on whether we want to consider something like this for the future.

Jdlrobson updated the task description. (Show Details)

Is there a task to follow this conversation? "Sign off steps" are not ticked so reopening to get that done.

Thanks, @Neil_P._Quinn_WMF. Does anyone know where to find the written-up research on this experiment? Or maybe who would have done it? Definitely would be interesting for our team.

oh dear, @Jdlrobson this was done at the request of the then mobile PM. I am baffled to hear the campaign code is still in there. You have my blessing to remove it. Let me know if I can help in any way.

Regarding the results, this was reported and discussed internally and apparently we didn't do the due diligence to complete the final report on Meta (I remember this was one of the last experiments the team ran before a change in product ownership). I can try and dig up the analysis in the coming days if that's helpful for future experiments.

ovasileva updated the task description. (Show Details)

documentation is completed and parties that might be interested in continuing the feature (or something like it) have been informed. I think we're good to go here.