@Tnegrin @Wittylama @Risker @Mike_Peel
Very pertinent to our conversation this morning.

One can already give feedback about a product from the Beta Feature page (there are links to feedback pages for each BF).
If the problem is rather that people shouldn't leave that page to be able to give feedback on the fly,
then the MediaWiki feedback tool could be invoked? (FWIW, such a tool can already be used within VE.)
But in this case, a project like Quicksurveys or some other kind of microsurveys that we've been asking for a while could be even more useful.

In T76573#1827971, @Elitre wrote:

One can already give feedback about a product from the Beta Feature page (there are links to feedback pages for each BF).

The purpose of this proposal is to allow users to provide feedback quickly in a way that it can inform the people in charge of each beta feature about their next steps. In particular, whether the feature in the current stage is (a) perceived as useful, (b) perceived as a good idea but is not ready yet, or (c) perceived as not useful.

The current free-form feedback pages are useful as a channel of communication but it is not easy to extrapolate the kind of support the feature has (e.g., are people supporting the feature considering it ready to move out of beta?) or whether people changed their mind after a feature had a major update.

If the problem is rather that people shouldn't leave that page to be able to give feedback on the fly,
then the MediaWiki feedback tool could be invoked? (FWIW, such a tool can already be used within VE.)

Not leaving the page to provide feedback is one technical aspect that would help users to provide their feedback faster. The underlying issue is both being able to provide feedback quickly but also to structure it in a way that reflects how well the idea and execution of it were received.

But in this case, a project like Quicksurveys or some other kind of microsurveys that we've been asking for a while could be even more useful.

It would be awesome if existing or planned products can support this one. The beta-feature rating system could be based on those survey systems but we need to consider the details of that integration. Ideally, the survey should appear by default for each new beta feature without additional work required, users should be able to change their response over time and the survey results correlated with beta feature events (e.g., allowing the developers to view the changes in opinion after a major feature was added).

Probably pertinent to this discussion is the fact that a user who has opted in to the " Automatically enable all new beta features" mode will not know when a new Beta feature has been added to their interface. Equally, if a feature's behaviour has changed since it was enabled the user might not know either.

This is where little explanatory popups could be extremely useful and, IMO, necessary. {I'm thinking of the popup that was recently deployed by the Visual Editor team for explaining the 'link' dialogue.]

If 'beta testers' was made an actual thing, with regular newsletters and a clear explanation of what was being tested (against what criteria for success), and what has changed recently, then I suspect that beta testers would be quite willing to have an obvious 'give beta feedback' button added to their interface.

In T76573#1841928, @Wittylama wrote:

Probably pertinent to this discussion is the fact that a user who has opted in to the " Automatically enable all new beta features" mode will not know when a new Beta feature has been added to their interface. Equally, if a feature's behaviour has changed since it was enabled the user might not know either.

That's T67182: BetaFeatures: Add a Notification for "A new Beta Feature is available to try"
and other related tasks: https://phabricator.wikimedia.org/maniphest/query/v6GWx2tcY9hA/#R
Sadly no-one has time to develop this currently, and as noted in some of those tasks, it's not simple. (It needs to interact with databases to determine who has which betafeatures enabled already, as well as just creating the new notification type(s), and knowing/inventing when to trigger sending them en-mass).

This is where little explanatory popups could be extremely useful and, IMO, necessary. {I'm thinking of the popup that was recently deployed by the Visual Editor team for explaining the 'link' dialogue.]

For reference, that was T108620: On user's first use of VE-MW, provide a couple of educational pop-ups for links and for citations with some toolbar highlights to trigger them.
I'm not sure if the MWEducationPopupTool used there, is currently used anywhere else?

A simple popup has also been considered, as an alternative to the proposed new Echo Notification types, in
T77366: Popup Guider for Beta Features - but I don't recall the outcome/details.

If 'beta testers' was made an actual thing, with regular newsletters and a clear explanation of what was being tested (against what criteria for success), and what has changed recently, then I suspect that beta testers would be quite willing to have an obvious 'give beta feedback' button added to their interface.

Hmm, that's probably a good idea, but just to devil's advocate... do you think a new newsletter is needed, or is an entry in Tech/News sufficient for the currently infrequent updates/releases/requests-for-feedback?
(Tangetially, I still hope that MediaWiki-extensions-Newsletter can be further developed and eventually deployed. :)

In T76573#1843206, @Quiddity wrote:

(Tangentially, I still hope that MediaWiki-extensions-Newsletter can be further developed and eventually deployed. :)

This is definitely happening. We were hoping for this quarter, but it looks like it will be in the next quarter -- see T110170: Goal: Deploy Newsletter extension in Wikimedia.

In T76573#1843206, @Quiddity wrote:

Hmm, that's probably a good idea, but just to devil's advocate... do you think a new newsletter is needed, or is an entry in Tech/News sufficient for the currently infrequent updates/releases/requests-for-feedback?
(Tangetially, I still hope that MediaWiki-extensions-Newsletter can be further developed and eventually deployed. :)

A talkapge newsletter would seem the most appeopriate method IMO (other things like a blog or email might work too I guess). I think a mention within the tech news might be worthy (just like is done when a new VE newsletter is released), but the point of what I'm describing is that this would be a frequent publication of practical changes and requests for testing - sent specifically to people who are opting-in to some kind of beta testing program. auch a program would be roughly analogous to the current "automatically enable all beta features" option, and perhaps signup to this newsletter would be an automatic part of joining the beta testing program.

The crucial difference between the current beta tools page and what I'm describing is that it should actually be used to, you know, beta test things! Currently it's more of a dumping ground of stuff with now clarify about whether the thing is being actively developed; what things have changed recently; what nature of the tool is considered to be 'beta'; what are the criteria for success or failure... Currently it's little-different to the "gadgets" page in user-preferences, just that all the gadgets are WMF-built.

Rather than split all the features up to different things people can opt-in/out of separately (like the list of gadgets), just make beta a single option - a user is either on the beta testing group, or not. [ that said. I can understand why massive things like VE could need to be separated out to their own specific opt-in]

The crucial thing about this 'beta testers' group is... The frequent communication about what is changing, what feedback is needed, what timeframe each feature is working to (some might be long-development and which are just a one week courtesy testing. And that means.... A talkapge newsletter! :-)

There's a related discussion on MediaWiki.org about beta features: https://www.mediawiki.org/wiki/Topic:Ss54yq1w3twctqsj
TheDJ and Jdforrester are suggesting that the Beta Features system would need to be completely rebuilt to be able to scale anyway and that it wouldn't catch the kind of bugs I'm suggesting it ought to.

Well that's depressing. It just means that the beta feature system is not actually ever going to be used consistently or effectively, it's at best a "WMF-only gadgets page" and at worst bloatware.

This ticket mentions many of the things that I have in the past suggested needed to change. Improved notifications, "getting started" flows, quantify the feedback that people have given.

But yes, the problem here is: Who will build it. To really do this properly, you could spend months on improvements and it will have to tie into a gazillion other things. I'm really happy however that there is time being spent on Flow Echo right now, that can really help this along as well at some point.

In T76573#1851916, @TheDJ wrote:

I'm really happy however that there is time being spent on Flow right now, that can really help along this as well at some point.

I would be very happy with Flow being built too - but development on it got cancelled, remember? They announced that it is in 'maintenance mode" a few months ago.

Instead, they're working on making a frameworks for "structured workflows" (like deletion debates, admin debates, 'did you know?' Nominations...) now. There was an excellent presentation at Wikimania Mexico. I think that's a really cool project idea, and really worthwhile. But it's sad that Flow got all the way to the "this is almost usable" stage and then got canned. :-(

sorry, i meant echo

In T76573#1851928, @Wittylama wrote:

In T76573#1851916, @TheDJ wrote:

I'm really happy however that there is time being spent on Flow right now, that can really help along this as well at some point.

I would be very happy with Flow being built too - but development on it got cancelled, remember? They announced that it is in 'maintenance mode" a few months ago.

Instead, they're working on making a frameworks for "structured workflows" (like deletion debates, admin debates, 'did you know?' Nominations...) now. There was an excellent presentation at Wikimania Mexico. I think that's a really cool project idea, and really worthwhile. But it's sad that Flow got all the way to the "this is almost usable" stage and then got canned. :-(

(Sidenote (please let's not tangent this task!) that Flow is not canned; it could be more accurately described as 'paused regarding major new feature development', whilst the small team puts more focus on cross-wiki notifications and other echo work, for the next few months. I'll try to clear up this ongoing confusion, onwiki/mailinglist, once a few more details about future plans/options are a bit firmer.)
And now back to Beta Features... :-)

Fair enough @Quiddity ;-) Cross-wiki notifications/watchlists is very important!

So, if I understand correctly, it is not possible to turn Beta features into a fully-fledged testing cohort inside the production wikis. That's a pity. However it doesn't mean it's not possible to use Beta Features more consistently and to greater effect.

The particular advantage of Beta Features is that it is on the live wikipedia (etc), rather than a separate testing wiki. Separate wikis for testing have their place, but the number of people who would be willing to test on them is greatly reduced. The value of being inside the normal wiki is:

• that people can test incidentally to their existing volunteer work, in their existing workflow.
• that people can feel like they're with the project to build the features, rather than being the customer upon whom new features are released at.

All of this goes back to the point I was making before; that a dedicated and regular announcement (mailing list, talkpage newsletter, blog, whatever) is crucial to building a sense of collegiality among people who are doing the beta testing, and also for directing their attention to new things that are either just added/changed, or just about to go into default for everyone.

While it would be nice to have little popups etc (the 'quick feedback' that this bug is referring to), I would say that more important (and technically easier to create) is regular communication with beta-users about what is happening and what role their feedback will play. Each feature needs to have a quantifiable target to reach before it goes into default (or is demoted back to being a gadget), and this beta testing group needs to be empowered to have some level of 'say' in whether that outcome.

T143173 looks related.