Dismiss this feature is basically the default behavior we have for quite all features: you can turn it anytime on and off in your preferences. Have a shortcut (cog icon) would be great.
Some experiences users may like to have it, since they can face technical problems while editing. Display it th them is also a good way to inform them of this new feature. Maybe we should think about a tour for experienced users, with the basics: 1/this is a new feature to ask for help while editing 2/ you can turn it off using this cog icon, or back on in your preferences.

@Trizek-WMF suggests that we turn the feature on for all accounts created after the feature is deployed, and just leave it on for them unless they dismiss it. @RHo, I noticed that the requirements specify that the feature cannot be dismissable. Can you tell us more about why that is important?

In T206716#4747523, @MMiller_WMF wrote:

@Trizek-WMF suggests that we turn the feature on for all accounts created after the feature is deployed, and just leave it on for them unless they dismiss it. @RHo, I noticed that the requirements specify that the feature cannot be dismissable. Can you tell us more about why that is important?

Leaving the feature on for all accounts after its deployment sounds good to me. I specified that it shouldn't be dismiss-able initially when we were considering having the feature be temporary and not something that could be turned back on, since we want something like help to feel as stable as possible for users. So long as we make it clear where a user needs to go to turn help pane back on, I am fine with making it a preference that can be toggled on/off for everyone.

Maybe we can have the shortcut to dismiss it only displayed when the users are autopatrolled (over 500 edits/30 days)? In any case, we should have a preference to toggle it on/off anytime.

In T206716#4753770, @Trizek-WMF wrote:

autopatrolled (over 500 edits/30 days)

Those kinds of autopromotion thresholds vary by wiki (and not all of them exist on all wikis). I think it's also important that a highly experienced user with a high global edit count but a low local edit count on e.g. cswiki doesn't get the help pane when they visit that wiki. So maybe it'd be better to set a threshold based on the user's global edit count and/or global account age.

@RHo @MMiller_WMF Should the help panel show when editing Talk pages? If so, should the "Include article title" checkbox reference the article page or the article's talk page?

On a related note, as a UX improvement I suggest including the article title below the checkbox so that it's very clear to the user what will get posted:

Following up on the preference: one way to proactively surface this and also get feedback from users would be to implement a "Feedback on this tool" dialog like we have elsewhere. The feedback dialog could tell users that there is a preference to disable the help panel.

Include talk pages is complicated. I would be for it if we can detect the namespace and provide links and context accordingly.

Include the title of the page (if we deploy it on Help namespaces) is IMO a good idea.

Concerning preferences and feedback, based on previous deployments, have a direct way to provide feedback and disable the feature is appreciated by experienced users. However, we are not targeting those users but people who will discover the interface as new (for most of them). Are we going to have some feedback then? I'm not sure, while people would not know that the panel is new. :)
If we decide to provide some feedback and opt-out links, I would go for a cog icon or something in the panel. People clicking on it would have then access to a two links menu: provide feedback and disable in preferences. @RHo may have a different opinion.

Agree with @Trizek_WMF about it being better to include Talk pages if we're able to detect the namespace and provide relevant guidance. On the other hand, we could just include Talk pages initially and monitor how much questions/help panel open triggers are coming from Talk pages before determining if there is a need to build this separate logic.

@kostajh - adding the title below the checkbox LGTM too!

As for including a link to turn off the help panel or link to feedback/info, we can add a 'cog' icon similar to the one on page preview pop-ups, but agree with @MMiller_WMF on T206717 that initially this doesn't need to appear on the feature itself.

One idea would be:

1. Display the help panel to all users, regardless of account age/edits etc, on the participating wikis
2. Newbies (using whatever criteria we decide for that, like account age/edits) do not see the cog for disabling the pane. But they could go to their preferences and switch it off if they are determined enoughl
3. Everyone else sees the cog which allows for one click disabling and providing feedback via an on-page modal (user clicks "Provide feedback" and a box opens for them, which posts on their behalf to the dedicated feedback talk page for this project).

That way:

1. Experienced users can provide feedback on the tool, use it if they want, and easily switch it off otherwise. We could include these users in our event logging and get some insights into how experienced users might use such a tool
2. We get insight into how many newbies dislike the tool enough to search their preferences to turn it off

I'm not very comfortable deploying a floating button into the users edit experience if they have no mechanism at all to switch it off

In T206716#4788999, @kostajh wrote:

One idea would be:

1. Display the help panel to all users, regardless of account age/edits etc, on the participating wikis
2. Newbies (using whatever criteria we decide for that, like account age/edits) do not see the cog for disabling the pane. But they could go to their preferences and switch it off if they are determined enoughl
3. Everyone else sees the cog which allows for one click disabling and providing feedback via an on-page modal (user clicks "Provide feedback" and a box opens for them, which posts on their behalf to the dedicated feedback talk page for this project).

That was my initial idea but what if a newbie asks how to remove it and if someone replies "click on the cog"?

@kostajh @RHo @Trizek-WMF @Urbanecm @revi -- I think we are discussing several issues in this task:

• Should the help panel be present on talk pages?
• Who should get this feature?
  • Only newbies?
  • All users?
  • As a beta feature?
• Should it be possible to disable it, and how?
  • In preferences?
  • In the feature itself?
• Should we include a feedback dialog?

Here are my thoughts, but let's continue to discuss:

Should the help panel be available when editing talk pages?

I think it should be available in all editing contexts for the users who have it turned on, whether that is in the main namespace, talk, user, user talk, etc. Newbies struggle to edit in all those contexts, and much of the help content about editing is relevant to all of them. I see the point about how it would be nice to include help contents around using talk pages, but that is an improvement we can handle another time when we know more about when and where people are using the feature. And then when the feature is automatically generating headers to post in the help desk, it can just include the namespace as part of the title, e.g. "Talk:Profiterole". Is there something I'm missing?

Who should get this feature / how to disable / including feedback dialog?

One thing we should keep in mind is that on each of Czech and Korean Wikipedias, there are about 250 active logged-in editors a day. I'm worried that if all those people suddenly have the help panel, the help desks could get flooded with dozens or hundreds of test questions, from editors who just want to know what the feature does. That would not be good. That's one of the reasons we were thinking of only giving it to new users.

But I think the use case of letting experienced editors try out the feature makes sense -- we don't people to create additional accounts just to try it out.

I also think it is smart to allow it to be turned off, and we would learn from the count of how many people turn it off.

Taking all that together, here is what I propose:

• There will be a user preference for the help panel's presence, located at this page. Not a "beta" feature.
• The preference is set on by default for all non-autocreated user accounts made after we deploy, that are in the treatment group.
• It is off be default for all other users.
• Users who do not have it on by default can go to preferences and turn it on. We expect that only a small number of experienced users who are following along with our conversations will do this.
• The feature can have a "cog" that links to the preference for turning it off.

Feedback dialog

I think having the feedback dialog option could be useful, but I don't think it's critical. Perhaps it could be on that final screen after submitting the question, near where the "More about this feature" link is. Is this a lot of work?

My thoughts:
Should the help panel be present on talk pages?
If the help panel invite and contents can't be displayed differently depending on the namespace, no.
The problem would be to display different links for a different use. And even, it would be easier to just add how to discuss than linking to pages.
I would focus on editing articles first.

Who should get this feature / how to disable
Everyone, since they can dismiss it directly from the feature ("cog preference").
I'm for it because:

• from all newcomers, some may want to opt-out
• from already existing accounts, some may appreciate to have a shortcut to get some help (but no Beta feature).

Concerning the workforce, that's something I can work on, with the ambassadors. They need to be prepared. We can offer a progressive rollout, but it is not up to us to decide.

In preferences and, if possible, in the feature itself.
Feedback dialog
I don't think that's critical either. Have a link to explain the feature is not important either, since we can imagine that people interested by it would go to the village pump to ask. To be considered if a deployment is made on a bigger wiki.

Time for my two cents :)

Should the help panel be available when editing talk pages?
+1 to @MMiller_WMF that it would be useful info to know at the start whether people want help from other namespaces like Talk or User, etc. It will give initial feedback as to what types of pages people are seeking help from (for example it is very useful to know if it does turn out that an inordinate about of questions are coming from Talk pages)

Who should get this feature?
+1 to @MMiller_WMF again regarding having it on by default for all new accounts, and an ability to turn on via preferences for existing editors who want to try it out. Since this is a new feature we are experimenting with specifically targeting to newcomers, at least at the start it makes sense to focus on this group as the target audience.

Should it be possible to disable it, and how?
Agree with emerging consensus on this thread about having the ability on Preferences to disable this feature.
Also fine with having a 'cog' to disable on the feature itself, but the cog will basically just take the user to their Preferences to disable it (in the same way clicking on the cog in Page previews takes the user to their preferences).

Feedback dialog
Happy to have it be a link on the last page under "More about this feature", or a section within the page that "More about this feature" point to. Or not at all if it is deemed not super critical, though I do think it is a relatively avenue to seek feedback from someone who has taken the time to submit a question.
At the very least having a "More about this feature" link seems beneficial for the target audience of new editors who likely do not know to go to Community places like Village pump.

After a team discussion today, we decided the following. I will also copy this into the task description.

Should the help panel be available when editing talk pages?

The help panel will be available in the editing context regardless of the namespace, with a couple notes:

• We will keep an eye on how often it is used in namespaces other than the main article space, to understand what help materials we will need to make it a better fit for those namespaces.
• We will encourage our Czech and Korean ambassadors to start thinking about help materials that are a good fit for the talk space.

Who should get this feature? How to disable it?

All users will have a user preference to toggle the help panel on and off. The only people who will have it on are the new users who create their accounts after launch who are also in the treatment (not control) group of our experiment. All other users could go to their preferences to turn it on. Anyone who has it on can turn it off by clicking a "cog" icon that will take them to the preferences page.

We also may want to turn the feature on for users who have 0 edits, but those users will be filtered out of our experimental data.

If the feature seems successful, and we aren't overwhelming the capacity of help desks to respond to questions, then we can turn the feature on more broadly for users who are not new.

Feedback dialog
Not needed for this feature.

In T206716#4798864, @MMiller_WMF wrote:

Anyone who has it on can turn it off by clicking a "cog" icon that will take them to the preferences page.

Ideally to the precise line (anchor on the preference id, if possible).

We also may want to turn the feature on for users who have 0 edits, but those users will be filtered out of our experimental data.

To do this I think we could have a configuration value for "Experiment launch date" (Jan 7 2019 at the moment) and then enable for all accounts that were created prior to that date, with 0 edits.

Anyone who has it on can turn it off by clicking a "cog" icon that will take them to the preferences page.

@MMiller_WMF @RHo please see my note in T209982#4801164, does that sound OK? I'd prefer to not have a link directly to Preferences#Editing where the user then has to figure out on their own how to disable the panel.

Change 477913 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Exclude namespace support, enable for older users

https://gerrit.wikimedia.org/r/477913

I think this can be up to @RHo (and I think we'll need designs for whatever route we go with).

@ovasileva -- we're talking about the workflow for a user to turn off the help panel feature we're building. You have a similar "cog" option in page previews that links to the preferences page. Did you all consider different workflows for turning off page previews? We're considering one in which the cog opens a little box telling the user what to look for once they get to the preferences page.

Change 477948 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Add settings cog with tooltip

https://gerrit.wikimedia.org/r/477948

Here's a simpler version for settings, with a tooltip to inform the user about what's going on. Doesn't solve the problem for mobile, but for desktop at least the user has some idea of what's going on.

Change 477949 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Add ability to exclude from namespaces

https://gerrit.wikimedia.org/r/477949

Change 477954 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Gradually enable for older zero edit users

https://gerrit.wikimedia.org/r/477954

Change 477913 abandoned by Kosta Harlan:
Help Panel: Namespace support, enable for older users, settings cog

Reason:
Split into separate patches

https://gerrit.wikimedia.org/r/477913

We also may want to turn the feature on for users who have 0 edits, but those users will be filtered out of our experimental data.

To do this I think we could have a configuration value for "Experiment launch date" (Jan 7 2019 at the moment) and then enable for all accounts that were created prior to that date, with 0 edits.

@Catrope suggested we run a maintenance script to enable it for all zero edit users created prior to launch date. I'm wondering if it might be preferable to more gradually enable the feature for the older zero edit accounts to help avoid flooding the help desk. cswiki and kowiki have 400,711 and 323,134 zero edit accounts respectively, about 10-12% (375k-400k) of these have user_touched (updated when settings are saved and when users log in) within the last 12 months, and about 1% (3.5-4.5k) with user_touched in the last 30 days.

In T206716#4801771, @MMiller_WMF wrote:

I think this can be up to @RHo (and I think we'll need designs for whatever route we go with).

@ovasileva -- we're talking about the workflow for a user to turn off the help panel feature we're building. You have a similar "cog" option in page previews that links to the preferences page. Did you all consider different workflows for turning off page previews? We're considering one in which the cog opens a little box telling the user what to look for once they get to the preferences page.

This sounds similar to the flow for logged out users: https://www.mediawiki.org/wiki/Page_Previews#Logged_Out_Users, where we have a little modal with some information on how to turn in on/off. I don't remember if we considered having something similar for logged-in users, but @Nirzar might know.

In T206716#4802330, @kostajh wrote:

Here's a simpler version for settings, with a tooltip to inform the user about what's going on. Doesn't solve the problem for mobile, but for desktop at least the user has some idea of what's going on.

@MMiller_WMF - I agree with @kostajh that it makes sense to have an explicit menu option rather than this tool tip or just taking users to Preferences:

And it also be a good place for the "More about this feature" page and "Send feedback" if we are going to have this in V1, but wouldn't be a top priority:

In T206716#4802372, @kostajh wrote:

We also may want to turn the feature on for users who have 0 edits, but those users will be filtered out of our experimental data.

To do this I think we could have a configuration value for "Experiment launch date" (Jan 7 2019 at the moment) and then enable for all accounts that were created prior to that date, with 0 edits.

@Catrope suggested we run a maintenance script to enable it for all zero edit users created prior to launch date. I'm wondering if it might be preferable to more gradually enable the feature for the older zero edit accounts to help avoid flooding the help desk. cswiki and kowiki have 400,711 and 323,134 zero edit accounts respectively, about 10-12% (375k-400k) of these have user_touched (updated when settings are saved and when users log in) within the last 12 months, and about 1% (3.5-4.5k) with user_touched in the last 30 days.

I propose we don't include those users just yet.

They won't be used in data analysis so I think the work associated with them (code complexity, deployment/maintenance script, help desk traffic) is basically a distraction from our main goals. Also, until we have some data, we don't know if this feature is beneficial or if it hurts in any way.

Can the link to disable the panel work directly, without forcing people to go to the preferences?
For now, it is not possible to link th a precise message and highlight it (I've documented that need T211204), which can confuse people.

In T206716#4803571, @Trizek-WMF wrote:

Can the link to disable the panel work directly, without forcing people to go to the preferences?
For now, it is not possible to link th a precise message and highlight it (I've documented that need T211204), which can confuse people.

No it should take them to do it in Preferences, since it would be problematic to enable someone to accidentally turn off the item they are clicking on, esp. without knowing where to go to switch it back on.

If we wanted to do it for them without requiring a trip to preferences, we could do something like page previews

However I'd view that as a nice-to-have considering all the other work we need to get done first.

@MMiller_WMF if you agree with T206716#4803500 please comment here and we can start building that.

Change 477949 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Add ability to exclude from namespaces

https://gerrit.wikimedia.org/r/477949

Is there really a strong case for having a way to disable the help panel button directly on it?

If I remember correctly we're trying to get people to ask for help while staying in context, not providing distractions and more ways to leave the editing context.

Also, in MediaWiki but especially with the editor open, there's about a hundred buttons and options available and none of them have a way to turn it off, you don't have to use it, that's all.

Also, in MediaWiki but especially with the editor open, there's about a hundred buttons and options available and none of them have a way to turn it off, you don't have to use it, that's all.

Agreed, but this is a bit more in your face as it follows you around the open editor.

Thanks, everyone, for talking this through. Taking all this together, I think it sounds like we should do @RHo's most recent design, with the second click to get to user preferences.

I think this:

• Makes it clear that if someone starts clicking in the cog, they are going to be leaving the help panel, and tells them where they are going to go.
• Sends them to preferences so that they know preferences exists.
• Gives us a place to put other links, like "More about help panel".

I will made a separate task for building this feature: T211400

@SBisson -- I agree that we should not actually launch with the feature on for users who registered before the launch date but still have 0 edits. I agree that it would be a distraction. I just think we should make it possible to do so if we ever want to. I will change the description to reflect that.

Change 477948 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Add settings cog with tooltip

https://gerrit.wikimedia.org/r/477948

Change 477954 abandoned by Kosta Harlan:
Help Panel: Gradually enable for older zero edit users

https://gerrit.wikimedia.org/r/477954

Settings cog is tracked in a separate task.

Checked in betalabs (except for the cog's links which is T211400). The specs in the phab task have been implemented. Below are just some observations for @MMiller_WMF to review.

(1) If a user entered an email address, but not yet confirmed it (not possible to do it now in betalabs) - the form will display the pre-filled 'Email address(optional)' field which may encourage users to enter a different email. Entering a different email won't change the existing email in the Preferences and it won't give a user any indication that their updated email does not have any effect.

Should we provide for this case or do not allow to enter a different email in the form (e.g. to treat it exactly as confirmed emails)?

(2) The text under "Email address (optionall)" field: "You'll receive an email confirmation link so that you can be notified of responses." and "You will receive an email notification when a response is posted" implies that unless an email address is confirmed, a user won't have notifications for responses, which is not quite true. If the Mention notification option (for new users it'll be a default option) is enabled, then echo notifications will come regardless of having a confirmed email address.

(3) It's rather difficult to see how the title of the help panel "Get help with editing" relates to "the editor help panel" in Preferences -"Enable the editor help panel". On the other hand, since at the beginning, the help panel won't be broadly introduced, it's fine.

Change 480290 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Set new email if API param does not match stored email

https://gerrit.wikimedia.org/r/480290

(1) If a user entered an email address, but not yet confirmed it (not possible to do it now in betalabs) - the form will display the pre-filled 'Email address(optional)' field which may encourage users to enter a different email. Entering a different email won't change the existing email in the Preferences and it won't give a user any indication that their updated email does not have any effect.

Just pushed a patch for this, thanks for catching!

If the Mention notification option (for new users it'll be a default option) is enabled, then echo notifications will come regardless of having a confirmed email address.

Hmm, that's not what I see. I just tested on testwiki with a flow mention and a wikitext ping, and I did not receive the email, even though "Mention" is enabled in my new user account prefs (with an unconfirmed email).

Change 480290 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Help Panel: Set new email if API param does not match stored email

https://gerrit.wikimedia.org/r/480290

@kostajh -- can you clarify the conditions under which you tested? What was the email status of your account? And what do you mean by "with a flow mention and a wikitext ping"?

@MMiller_WMF I created a new account on testwiki with an email address and did not confirm the address. I received the email to confirm, so I know I'm receiving at least that email. Then I pinged this account with wikitext here https://test.wikipedia.org/wiki/User:KHarlan_(WMF)/sandbox and via a Flow mention here https://test.wikipedia.org/w/index.php?title=Topic:Ult2yv7e0vbonuxf&topic_showPostId=uqkqrvnh3yc321t4#flow-post-uqkqrvnh3yc321t4 I did not receive an email in my Notificationtest account for either of those.

@kostajh -- I just re-read this. @Etonkovidova is pointing out that people who do not have a confirmed email will still receive echo notifications on wiki (not email notifications), and that therefore the text might be misleading because it could make it seem like the only way to find out about the response is through email. I think that the language is okay, because we believe that users are much more likely to notice notifications via email than through echo alerts on wiki. I'm afraid to clutter and confuse by adding additional caveats to the sentence. Is this what you were getting at @Etonkovidova?

@MMiller_WMF

In T206716#4839615, @MMiller_WMF wrote:

@kostajh -- I just re-read this. @Etonkovidova is pointing out that people who do not have a confirmed email will still receive echo notifications on wiki (not email notifications), and that therefore the text might be misleading because it could make it seem like the only way to find out about the response is through email. I think that the language is okay, because we believe that users are much more likely to notice notifications via email than through echo alerts on wiki. I'm afraid to clutter and confuse by adding additional caveats to the sentence. Is this what you were getting at @Etonkovidova?

Right, but reading thinking more about it and reading the text "If you add an email address to your account, you'll receive an email confirmation link so that you can be notified of responses." - it sounds ok. It'd be nice to introduce to a user echo notifications, but the Help panel has its obvious space limitations.

@RHo - Please review the following - when a question failed to be posted, a user woul see:

In T206716#4853843, @Etonkovidova wrote:

@MMiller_WMF

In T206716#4839615, @MMiller_WMF wrote:

@kostajh -- I just re-read this. @Etonkovidova is pointing out that people who do not have a confirmed email will still receive echo notifications on wiki (not email notifications), and that therefore the text might be misleading because it could make it seem like the only way to find out about the response is through email. I think that the language is okay, because we believe that users are much more likely to notice notifications via email than through echo alerts on wiki. I'm afraid to clutter and confuse by adding additional caveats to the sentence. Is this what you were getting at @Etonkovidova?

Right, but reading thinking more about it and reading the text "If you add an email address to your account, you'll receive an email confirmation link so that you can be notified of responses." - it sounds ok. It'd be nice to introduce to a user echo notifications, but the Help panel has its obvious space limitations.

@RHo - Please review the following - when a question failed to be posted, a user woul see:

hi @Etonkovidova – Looks OK to me, so long as selecting the "Wikipedia: Help desk" link opens the link in a new tab/window?

@RHo - Please review the following - when a question failed to be posted, a user woul see:

hi @Etonkovidova – Looks OK to me, so long as selecting the "Wikipedia: Help desk" link opens the link in a new tab/window?

Thx! Yes, the link "Wikipedia: Help desk" is correct.