we drafted a usability protocol to text the current ux (internal google doc) and launched a pilot study to test protocol too with userlytics

we completed the pilot study, refined the protocol with input from product and design research, and launched an unmoderated study with 5 participants.

we've got the results (4 out of 5), and we've started to watch recordings and synthesizing findings... for 1 participant the test didn't work properly and we've requested a new participant

we've not completed the study, and drafted a first report (internal google doc)

we' started to explore a few ideas (internal figma file), and initiated a group conversation around the main benefits connected to account creation (internal slack thread)

we've created an internal miro board, and prompted a few stakeholders across P+T to share async feedback ideally by this wed, nov 5 end of their day..

I think the particular text and interface we would want to modify is created in includes/EditPage/EditPage.php#2892, assuming it is ok for Growth engineers to work on that area of the code, I don't know how we'd conduct an experiment for a core feature. Being xLab/MetricsPlatform an extension the setup described in the docs https://wikitech.wikimedia.org/wiki/Experimentation_Lab/Conduct_an_experiment#Code is not directly actionable. Has xLab been used for an A/B test in core before? I vaguely remember Sam mentioning a PHP version A/B test setup, but Idk if that example is even relevant to what we want to do here. cc @mpopov @phuedx

Alternatively, we could explore some ways to make the intro messages interface more customizable by extensions, it seems there's already EditPage::showEditForm:initial that goes on the lines of altering what's displayed as introduction messages in the editor but I'm not sure that hook is enough to modify what we want here. Does this sound doable? Or do you have any alternative ideas? cc @matmarex @Michael @Urbanecm_WMF

In T407497#11341333, @Sgs wrote:

I think the particular text and interface we would want to modify is created in includes/EditPage/EditPage.php#2892

There are at least two interfaces you might be interested in. (See docs and screenshots for each editor here: https://www.mediawiki.org/wiki/Editor)

1. I looked at the internal Google docs linked by Amin above (it would be nice if they could be made public), and they feature screenshots of MobileFrontend's wikitext editor and the mobile version of VisualEditor.
   • The message used by them both is generated here (MobileFrontend EditorOverlayBase).
   • Try it out live here (open in a private window to see the anon warning).
2. The code you linked to generates the message shown e.g. in the default MediaWiki editor, the 2010 wikitext editor and desktop VisualEditor.
   • The message is actually generated here (core EditPage/IntroMessageBuilder), the code you linked ultimately calls that function, and that's where you'd have to add a hook or whatever.
   • You can only see this message on mobile if you try to do something that isn't supported by the mobile editor, e.g. undoing an edit – try it out live here (open in a private window to see the anon warning).

The text in both cases in almost the same, but the messages are generated in different places, and use different message keys.

I don't know how we'd conduct an experiment for a core feature. Being xLab/MetricsPlatform an extension the setup described in the docs https://wikitech.wikimedia.org/wiki/Experimentation_Lab/Conduct_an_experiment#Code is not directly actionable. Has xLab been used for an A/B test in core before? I vaguely remember Sam mentioning a PHP version A/B test setup, but Idk if that example is even relevant to what we want to do here. cc @mpopov @phuedx

Per above the experiment code might actually go into MobileFrontend, but if you needed to do something in core, I think you could follow the same process – either add a hook in the right place in core, and then put the bulk of your code in the WikimediaEvents extension; or put it directly in core guarded by ExtensionRegistry::getInstance()->isLoaded( 'EventLogging' ) or something like that. The latter is icky, but probably acceptable for a short-lived experiment.

Alternatively, we could explore some ways to make the intro messages interface more customizable by extensions, it seems there's already EditPage::showEditForm:initial that goes on the lines of altering what's displayed as introduction messages in the editor but I'm not sure that hook is enough to modify what we want here. Does this sound doable? Or do you have any alternative ideas? cc @matmarex @Michael @Urbanecm_WMF

Aside from the fact that you might need to do this experiment in MobileFrontend, per above, I think putting some hooks in IntroMessageBuilder would make sense. Ideally they'd go there and not directly in EditPage, since it would be good to refactor it and delete that class one day, and having more hooks would make it harder.

This is very helpful @matmarex, thank you! I will dive into the details to try to give a rough plan and estimation to each of the proposed interventions.

last week, we've tested 3 variants of the logged-out warning message with 15 participants (5 per variant). the unmoderated test observed participants’ reactions to different timings and visual treatments of the authentication prompt when logged-out users attempt to edit (full report)

based on the latest, and previous findings we've drafted this design proposal:

• remove the logged-out warning that is displayed as soon as people open the editor
• display a "logged-out reminder" while the editor is loading, and once is loaded
• display a "pre-publish dialog" when logged-out editors tap on .ve-ui-toolbar-saveButton

Follow up spike: T411880: Engineering Spike: technical feasibility of proposed Logged-Out Warning Message UX