Page MenuHomePhabricator

RFC: Standard method for feature-management in skins/extensions
Open, HighPublic

Description

Preamble

As part of the Wikimedia Foundation’s Medium-term plan, the Readers Web team will be making significant improvements to the desktop experience of Wikimedia projects.

The Desktop Improvements project is divided into a set of many stages. Each of those followed by a rollout to testing wikis. We are building a new look&feel for Wikimedia projects. One of our requirements is to keep the old look&feel for people who do not want to use Desktop improvements while we still work on it.
Once the project is done and we solve all problems with gadgets/users scripts/templates. We plan to migrate all the features we develop into default experience. Before that happens, everything has to be feature-flagged and visible only on some testing wikis. Additionally, we would like to provide an opt-out on testing wikis and opt-in on all wikis.

The proposal here is to agree on two things:

  • the way on how to manage new, significant changes that are feature-flagged. We should be able to quickly A/B test those changes/allow the user to opt-into "improved mode" to see new UI and later promote those to as default experience. We would like to use Feature Management from Mobile Frontend. It fits our needs nicely and solves most of the problems stated in this RFC.
  • where to keep the new code that could help us tackle this problem

Problem statement

Currently, there is no feature management in MediaWiki Core. The code has to handle its visibility state by using $config object and some additional checks that may have to be hardcoded in multiple places. It is not a problem when we want to handle only a single feature. The problem arises when we want to:

  • disable/enable multiple features at once
  • do incremental rollouts where feature depend upon each other and only one feature is available for public
  • promote some parts of functionality into default experience but still keeping rest disabled
  • A/B test some features for a short time
  • allow one feature to be available only if another feature is available
  • supporting dev environments and 3rd parties in a proper manner (dev should be experimental, 3rd party gets finished features)
  • handling configs and opt-in/opt-outs can become a complex set of if statements in many places of the codebase.

Most of this stuff can be hardcoded, but when we want to do similar things for different features, we find ourselves writing the same code over and over again. For example, the A/B testing procedure looks like that:

  • a) write and deploy some code to support A/B test
  • b) SWAT the config to enable A/B test
  • c) disable the A/B test code by SWAT
  • d) remove the A/B testing code

Existing solution

The Feature Management implemented in MobileFrontend already tackles some of those points. We would like to promote Feature Management so it can is widely used by different extensions/teams.
The idea behind Feature Management is that we have 2 main components - a list of all available features and a list of all available modes. On top of that, we have a manager that has access to all available features, mode, and config.
In configs, we store configuration per each feature. Each feature config contains a list of modes it's enabled/disabled in, e.g.:

"MinervaAdvancedMainMenu": {
    "value": {
        "base": false,
        "beta": false,
        "amc": true,
    }
},

or

"MinervaHistoryInPageActions": {
    "value": {
        "base": false,
        "beta": false,
        "loggedin": true
    }
},

The attached config means that AdvancedMainMenu feature is available only in the AMC mode, but not in beta nor default mode, and the HistoryInPageActions is available only for logged in users. In the system
we only call $featuresManager->isAvailableForCurrentUser( 'AdvancedMainMenu' ) and everything happens under the hood. When we want to A/B test some feature, we need to provide an A/B test mode, sth like:

"MinervaHistoryInPageActions": {
    "value": {
        "base": false,
        "beta": false,
        "experiment_vector_improvements": true
    }
},

and then the experiment mode would bucket the user and based on some criteria return if feature is available or not (this part is not implemented yet).

Modes are specified in code ( implementations of IUserMode). Currently, we have StableMode, BetaMode, AMCMode and LoggedInMode. Moving features between modes are as simple as just changing the configs. No code is required. Adding new mode narrows down to creating a new class that implements IUserMode and registering it in ServiceWirings file.

You can see the existing solution here: https://github.com/wikimedia/mediawiki-extensions-MobileFrontend/tree/c30112068cc95c4523abd472ee0a644aecdbb9a9/includes/features

Project requirements

The Feature Management (or anything that we come up with) has to

  • support feature flagging some parts of the app
  • provide a way to bundle features into packages user can opt into
  • provide an easy way to depend not only config flags but also on the ContextSource (namespace/logged_in users etc.)
  • provide a unified way to A/B test features on the backend side
  • supporting default configs for dev/3rd party/prod environments
  • provide an easy way to promote things to stable without us having to edit code

After a quick chat with other teams, it looks like there is a need for such system as many projects involve complex logic for features visibility.

Proposals

The engineers of the Readers Web team came up with five proposals. Each option has many pros and cons.

Proposal 1: No framework (config-based conditionals)

Instead of implementing the Feature Management system, we can utilize what we currently have in MediaWiki. Config arrays have some merging strategies that could help us solve some such problem.

Pros:

  • nothing to write, we already have config objects
  • the best code is no code

Cons:

  • bit messy, need to remember many config options, might have to write functions that do checks across multiple config flags and/or user options
  • need to write new code every time we want to change feature visibility (A/B test it, enable for logged in, enable as default experience)
  • easy to make mistakes
  • difficult to unit test as each feature will have it's own logic when it's visible/not
  • merging strategies in config files can be difficult to understand
Proposal 2: Use or extend BetaFeatures

The BetaFeatures extension is designed to serve such a purpose. Not all things is supported by BetaFeatures, and we would have to implement Feature Sets and handle anonymous users (as currently, BetaFeatures can handle only logged-in users). Another problem we face is that the core functionalities of Vector will depend upon an extension which, in our opinion, is not the optimal solution.

Pros:

  • it feels like BetaFeatures is a great place for such code

Cons:

  • another dependency making Vector and other extensions we're going to touch depend upon BetaFeatures.
  • it needs lots of work. Currently, BetaFeatures supports only opt-in for a single feature. There are no feature sets, no modes.
  • it feels like a/b testing doesn't fit into BetaFeatures.
  • supporting instances without BetaFeatures extension is tricky. Desktop Improvements outcome is going to be default experience. Tangling Desktop Improvements with an extension that might not be available on 3rd party makes it bit more complex
  • BetaFeatures supports only logged-in users, we want to enable some features for anon users
Proposal 2: Local to Vector

Most of our work circles around Vector skin, therefore it makes sense to implement the code supporting our changes in Vector. There are many other extensions (like UniversalLanguageSelector) that inject UI elements to Vector, and we would like to amend those. It means that those extensions will have to be aware of special Vector handling (the improvements mode). Therefore it creates a dependency upon Vector which is not an optimal solution.

Pros:

  • fast and easy
  • we don't have to worry about different parts of the ecosystem

Cons:

  • code duplication with MobileFrontend
  • there might be problems when it comes to other extensions, as other extensions might depend upon Vector (by using Vector Feature Management classes/interfaces)
  • we will provide lots of useful code that most probably won't be used anywhere else
  • code won't be available for other teams to use it
Proposal 3: Implement in MediaWiki core

The Vector skin is the default skin shipped with MediaWiki core. We would like to minimize Vector dependencies to a minimum. Therefore if we want to create a reusable solution, it should go into something that Vector already depends.
Feature Management code feels like a bit more complex Config structure that additionally performs various checks before it returns true or false.

Pros:

  • Feature Management can be unified and used across Vector, Minerva, MobileFrontend, and other extensions/skins. Our work will touch not only Vector but also other extensions.
  • can lead to simplifying extensions/skins code as most of the responsibility will be held by Core.
  • later could be used by different projects. Therefore code is expected to evolve
  • it feels like this should be an integral part of MediaWiki
  • no external dependencies skins/extensions

Cons:

  • longer review cycle
  • what to do with BetaFeatures extension - maybe FeatureManagement could use BetaFeatures as storage?
Proposal 4: Implement as standalone library (Composer)

We also explored an option to create an external library and load it with the composer in all extensions/skins we're going to amend. Sadly this option raised lots of uknowns, and it is an area we could explore more.

Pros:

  • Easy to start and start writing code
  • clear separation between FeatureManagement library and code that uses it

Cons:

  • can be confusing to bind with MediaWiki ecosystem (Hooks/ServiceWirings etc.)
  • at the end it cannot be a standalone library as it needs to utilize MediaWiki ecosystem

Author recommendation

Readers Web had a long discussion on how to tackle this problem, and we would like to head with Implement it in core (Proposal 3). We already tried to support Feature Management in Minerva, which can run without MobileFrontend ( in desktop mode), and we encountered many small bumps on the way. The Feature Management feels like something Core should support/provide, and Beta Features extensions feel like a "storage" space for user opt-in/opt-outs.

The current solution is an adaptation of previous Stable/Beta mode, and it has some quirks which we need to solve before porting it into Core. Also, we learned we could not depend upon RequestContext in ServiceWirings file, and the system has to pass current IContextSource when performing feature checks. With those lessons learned, we genuinely think we can provide a fantastic piece into the core MediaWiki that could be used by different teams across multiple projects.

Before we start doing that, we would like to reach a wider group and ask for comments and suggestions.

Event Timeline

Restricted Application added subscribers: Masumrezarock100, Aklapper. · View Herald TranscriptJan 15 2020, 11:08 AM
daniel moved this task from Inbox to Under discussion on the TechCom-RFC board.Jan 15 2020, 9:41 PM

@polishdeveloper Is it implied that the feature in question would be (mainly) developed and maintained long-term by Reading-Web? If not, is there roadmap/stewardship in place with another team?

I'm asking to determine how to priotise this as we generally prefer not to invest more heavily in the discussion until resourcing etc has been determined. Thanks!

@Krinkle I don't think there is a roadmap/stewardship in place for this project. We found a need for such solution, I also did a quick research and looks like other teams are facing similar problems as we start to feature flag and A/B test many things. This is an idea to centralize the common requirement we, Readers Web have (maintaining multiple modes) and make it handy for other teams. I can definitely help with maintaining such project in my volunteer time.

@Krinkle I'd like to follow-up on what I wrote earlier. I checked with Readers Web, and we agreed that we currently maintain the Feature Management in MobileFrontend. We're going to do something similar in Vector, therefore we will have to maintain it also in Vector. Therefore there is no difference for us between maintaining it in MobileFrontend/Vector or maintaining it in the MediaWiki Core.

TL;DR; - Readers Web will maintain the Features Management in core.

Jdlrobson moved this task from Incoming to Needs Prioritization on the Readers-Web-Backlog board.
ovasileva triaged this task as High priority.Feb 19 2020, 4:34 PM

Adding the priority from the team's side

kchapman added a subscriber: kchapman.

Adding CPT for visibility

Krinkle renamed this task from [RFC] Port MobileFrontend Feature Management into core to RFC: Standard method for feature-management in skins/extensions.Feb 19 2020, 10:06 PM
Krinkle updated the task description. (Show Details)
WDoranWMF added a subscriber: WDoranWMF.

@kchapman Should we track this under an initiative?

Tgr added a subscriber: Tgr.Feb 26 2020, 3:11 AM

Sounds great, a feature flag system in core would be quite useful.

One thing we needed several times in GrowthExperiments is a hidden user option shadowing a site setting (so that community members involved in development can test the feature before it is publicly enabled). I guess that would be a UserMode here?

@Tgr yes, so in this idea you would have - the UserMode called ( GrowthCommunityMemeber ).
Then you would have multiple features, that are not available to any mode, only to growthCommunityMemeber. Once User opts into growthCommunityMember (or someone opt's that person) user will get all features available on that mode.

We started some work on this project, to have some working solution till this RFC is closed. Please check T244481: Provide basic FeatureManagement in Vector codebase - and the gerrit patch: https://gerrit.wikimedia.org/r/#/c/mediawiki/skins/Vector/+/572323/
Instead of UserMode I started using Set, because some modes are not typical user modes - for example an A/B test or an experiment. It's not a UserMode - it's more a set of features that is available in given context.

So, to your comment, you would have to define:

$featureManager->registerSet( new UserSelectableSet( 'GECommunityMemeberSet', {OPTION_NAME});

or just define a GECommunityMemberSet that extends UserSelectableSet and in __construct() it passes the name, and the option_name. The name could be defined as const so it's easily accessible in the codebase.

How to use it

You must define a feature (so system knows there is a feature with given name). This can be done by $featureManager->registerFeature();
Then once feature is registered - feature is there but is not available to anyone. To make it available only for a community member you would define the feature config as:

"SomeAmazingFeature" => [
  "GECommunityMemeberSet" => true
]

which means that feature is available only for people who opted into GECommunityMemeberSet. Once you decide to make this feature available to everyone you just change the config to

SomeAmazingFeature => [
  "loggedin" => true
]

and immediately everyone logged in will get that feature, no code changes are required.

Now, how to access those things, in code you would do:

a) to check if user has a feature:

MediaWikiServices::getInstance()->getService('FeatureManager')->hasFeature( $FEATURE_NAME, $contextSource);

b) to check if user opted into mode/set

MediaWikiServices::getInstance()->getService('FeatureManager')->isSetActive( $SET_NAME, $contextSource);

And the last bit, we're still thinking on how to name the UserMode, I came up with the name Set, as UserMode was a bit too narrow. We're open for ideas for a better name.

Would a meeting with Tech Com or a public IRC meeting be useful at this point?

I'm more than happy to join any meetings, explain the idea behind Feature Management and try to answer all questions.

WMDE-leszek rescinded a token.
WMDE-leszek awarded a token.
WMDE-leszek added a subscriber: WMDE-leszek.
awight added a subscriber: awight.Mon, Mar 23, 10:35 AM
WMDE-Fisch added a subscriber: WMDE-Fisch.