Page MenuHomePhabricator

[Spike, 1 day] Analyze/analyse usage of base in mobile feature management
Closed, ResolvedPublic2 Estimated Story Points

Description

Background

In T388036, we were surprised to learn that the behavior of base in our feature management system is different than common understanding. As a result, there is at least one known case of us rolling out a feature with the configuration base: true and loggedin: false, which is inadvertently targeting logged in users

We want to check our production configurations and document all the cases where base is true, but the configuration suggests it was not intended to target everyone

Additionally, we want to document all cases where base is true regardless, as these would potentially be impacted by a change implemented as part of T388202

User story

As a member of the web team, I want to be confident that my features are configured properly and rolled out to the correct audiences

Requirements

[ x] production configurations are audited for cases where base is true
[x ] state of the world is documented in this ticket, with a team discussion accompanying any similar cases where our rollout is presumed incorrect (base is true and other modes are false)
[ x] additionally, an overall view of the usage of base is provided in the ticket, which will inform and unblock work on T388202
[x ] followup tickets are created if necessary

Acceptance criteria

  1. We are confident that the full impact of this bug is understood
  2. We have a plan and path forward for anything that comes up as part of this investigation
  3. T388202 is able to be scoped and estimated, with the understanding of what the impact of changing the existing definition of base would have

Rollback plan

No code is being written for this task, so it's not something that can be rolled back, per se

Event Timeline

@Jdlrobson-WMF it's not Friday morning, but here's the other ticket - apologies for the delay

Jdlrobson-WMF renamed this task from Analyze/analyse impact of logged out user bug to [Spike, 1hr] Analyze/analyse impact of logged out user bug.Mar 10 2025, 9:40 PM
Jdlrobson-WMF changed the task status from Open to In Progress.
Jdlrobson-WMF triaged this task as High priority.
Jdlrobson-WMF set the point value for this task to 1.
Jdlrobson-WMF moved this task from Incoming to Sprint Backlog on the Web-Team board.

I feel like this is more than an hour, but I don't want to split hairs haha - in any case 1pt is probably right

Jdlrobson-WMF renamed this task from [Spike, 1hr] Analyze/analyse impact of logged out user bug to [Spike, 4hr] Analyze/analyse impact of logged out user bug.Mar 10 2025, 10:08 PM

Okay upped to 4hrs. I don't think we should spend any more time beyond that.

SToyofuku-WMF renamed this task from [Spike, 4hr] Analyze/analyse impact of logged out user bug to [Spike, 1 day] Analyze/analyse usage of base in mobile feature management.Mar 13 2025, 11:34 PM
SToyofuku-WMF updated the task description. (Show Details)
SToyofuku-WMF changed the point value for this task from 1 to 2.

Change #1130771 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[operations/mediawiki-config@master] Web features should not be ambiguously configured

https://gerrit.wikimedia.org/r/1130771

I wrote a test to evaluate keys which might be impacted.

The only case where we enable a feature for anons but not logged in is wgMinervaDonateBanner.

However, if we expand the script to identify configuration where base is defined as true but loggedin is ambiguous (e.g. has not been defined in configuration) we get several other matches. I wrote a patch to test for that and addressed the offenders above. I think this is a necessary precursor before redefining the behaviour on mobile to match desktop.

Change #1130771 merged by jenkins-bot:

[operations/mediawiki-config@master] Web features should not be ambiguously configured

https://gerrit.wikimedia.org/r/1130771

Mentioned in SAL (#wikimedia-operations) [2025-03-26T22:09:00Z] <toyofuku@deploy1003> Started scap sync-world: Backport for [[gerrit:1130771|Web features should not be ambiguously configured (T388445)]]

Mentioned in SAL (#wikimedia-operations) [2025-03-26T22:15:34Z] <toyofuku@deploy1003> toyofuku, jdlrobson: Backport for [[gerrit:1130771|Web features should not be ambiguously configured (T388445)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Change #1131469 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[operations/mediawiki-config@master] Restore simplified watchlist for logged in users

https://gerrit.wikimedia.org/r/1131469

Mentioned in SAL (#wikimedia-operations) [2025-03-26T22:32:41Z] <toyofuku@deploy1003> Finished scap sync-world: Backport for [[gerrit:1130771|Web features should not be ambiguously configured (T388445)]] (duration: 23m 41s)

Change #1131469 merged by jenkins-bot:

[operations/mediawiki-config@master] Restore simplified watchlist for logged in users

https://gerrit.wikimedia.org/r/1131469

Mentioned in SAL (#wikimedia-operations) [2025-03-26T22:54:16Z] <toyofuku@deploy1003> Started scap sync-world: Backport for [[gerrit:1131469|Restore simplified watchlist for logged in users (T388445)]]

Mentioned in SAL (#wikimedia-operations) [2025-03-26T22:58:41Z] <toyofuku@deploy1003> jdlrobson, toyofuku: Backport for [[gerrit:1131469|Restore simplified watchlist for logged in users (T388445)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2025-03-26T23:06:45Z] <toyofuku@deploy1003> Finished scap sync-world: Backport for [[gerrit:1131469|Restore simplified watchlist for logged in users (T388445)]] (duration: 12m 29s)

I have reviewed this three different times now and I'm confident this is satisfactory - thank you for being so thorough!!