Kim and I had a great pairing session, and we think we've figured this one out!

The TL;DR is that it's unexpected behavior relating to our feature management system 🙃

so the current config in beta is

'wgMinervaDonateBanner' => [
	'default' => [
		'base' => true,
		'loggedin' => false,
		'amc' => false,
	],
],
'wgMinervaDonateLink' => [
	'default' => [
		'base' => false,
		'loggedin' => true,
		'amc' => true,
	],
],

but what we're seeing on beta is for logged in users, they get both the banner and the link, while for logged out they are correctly shown only the banner. After experimenting a little bit with the config, we were able to isolate that it is indeed only the banner config that is behaving incorrectly (true for both logged in and logged out), while the link is properly only shown for logged in. We explored a bunch of theories around why this could be, but eventually ended up digging into the feature manager code to understand how this all works (after remembering that this is code we wrote and not mediawiki core functionality like I often think it is)

The method in question is this one, which loops over a list of accepted "user modes" to determine whether any of them are enabled and, if so, if the feature is available in that particular mode. If any modes satisfy these two requirements, then the feature is considered available for a particular user. In the case of the donate link, the user is logged in, and the loggedin key is set to true, so this feature is considered "available" for a logged in user

What was surprising to us is that base is also a user mode like any other, specifically "stable user mode", which is defined as any user that is not a beta group member (which as I understand it, now refers to everybody). Since base is set to true in the config, the feature is considered available for all users regardless of logged in status, and the loggedin key is ignored because we have a true value in the list of available user modes

That all is to say, the reason that we're displaying the donation banner for logged in users is because base is set to true, and this can't be overridden to false by a subsequent key like we had previously assumed

As for what this means, we have a couple of options:

1. We can remove the feature/config code related to the donate banner, and set it at the template level to be equal to the inverse of the donate link. We checked with Olga and she said there should never be a case where we show both the banner and link or neither, so this should work fine - the biggest downsides here are that if there ever is a case where we want to turn both off it's not easy, and it's a bit less obvious how things are working in the code
2. We can write a loggedout user mode. Similar to how loggedin is currently implemented, if we change the MinervaDonateBanner config to match the MinervaDonateLink config, except with loggedout set to true, this should allow us to target only logged out users as we intended. The biggest downside here is that user mode currently doesn't exist, so we would have to write it

Either way, we should probably think as a team about how to better document/disseminate the fact that base doesn't behave the same as default in mediawiki config, because it seems like we all at least kind of thought it did

It could also be that there is another option!! This is what we were able to come up with as paths forward

The method in question is this one, which loops over a list of accepted "user modes" to determine whether any of them are enabled and, if so, if the feature is available in that particular mode. If any modes satisfy these two requirements, then the feature is considered available for a particular user. In the case of the donate link, the user is logged in, and the loggedin key is set to true, so this feature is considered "available" for a logged in user

This does sound like a bug. The modes are meant to be mutually exclusive. Interestingly looking at wmf-config/skin-Minerva.php in configuration.. it seems we never intentionally enabled the donate link in production... so the donate link showing there is likely unintentional! Can you raise a ticket.. let's get this behaviour fixed before it trips us up again.

After experimenting a little bit with the config, we were able to isolate that it is indeed only the banner config that is behaving incorrectly (true for both logged in and logged out), while the link is properly only shown for logged in.

Interesting. I was unable to replicate this locally as I had $wgMFEnableBeta = true; set locally. When I remove that line I can indeed replicate this.. I wonder if this is another signal that our beta code needs removing as proposed in T385509. Any idea why that might be happening?

It could also be that there is another option!! This is what we were able to come up with as paths forward

We need to be able to progressively roll out.. I think I have an option option to propose: we check logged in status along with the feature flag e.g.

			'data-donation-banner' => $this->skinOptions->get( SkinOptions::SHOW_DONATE_BANNER ) &&
				$this->getUser()->isAnon(),

Talked in standup. Next steps are:

• @KSarabia-WMF to workaround this ticket by updating code to assert the user is anonymous. Let's add an inline comment pointing to the URL of Steph's comment above for reference
• @SToyofuku-WMF to create a bug report that our expectation is that we should be able to use this to ship features only to logged out users, using the existing bug that the donation link is unexpectedly being shown to logged in users as an example.
• @SToyofuku-WMF to create as spike for checking if any of our other feature flags are impacted (e.g. are true for logged out but false for logged in)

Change #1125251 had a related patch set uploaded (by Kimberly Sarabia; author: Kimberly Sarabia):

[mediawiki/skins/MinervaNeue@master] Donation banner feature flag fix

https://gerrit.wikimedia.org/r/1125251

Change #1125251 merged by jenkins-bot:

[mediawiki/skins/MinervaNeue@master] Donation banner feature flag fix

https://gerrit.wikimedia.org/r/1125251

Test Steps

Status: ❌ FAIL
Environment: cawiki
OS: macOS
Browser: Chrome
Device: MS

Test Case 1: Verify that logged-in users do not see the donation banner

1. Log in to a Minerva-skinned wiki.
2. Visit any page with the donation banner enabled.
3. Open the sidebar.
4. ✅ AC1: Confirm that the donation banner is not visible.

Test Case 2: Verify that anonymous users still see the donation banner

1. Open an incognito window or log out.
2. Visit a Minerva-skinned page with the donation banner enabled.
3. Open the sidebar.
4. ❌ AC2: Confirm that the donation banner is displayed correctly.

@Edtadros please repeat the test. Fixed now! Thanks in advance!

Test Steps

Status: ❌ FAIL
Environment: cawiki
OS: macOS
Browser: Chrome
Device: MS

Test Case 1: Verify that logged-in users do not see the donation banner

1. Log in to a Minerva-skinned wiki.
2. Visit any page with the donation banner enabled.
3. Open the sidebar.
4. ❌ AC1: Confirm that the donation banner is not visible.

@Jdlrobson-WMF , Now both donation banners are gone.

Test Case 2: Verify that anonymous users still see the donation banner

1. Open an incognito window or log out.
2. Visit a Minerva-skinned page with the donation banner enabled.
3. Open the sidebar.
4. ✅ AC2: Confirm that the donation banner is displayed correctly.

❌ AC1: Confirm that the donation banner is not visible.

Yes for logged in users that's expected. We do need to clarify the exact behaviour during T388438: Gradual Rollout - Donate Button Deployment as it wasn't clearly specified. Per the A/C we only care about the donate banner not displaying. We haven't specified donate link behaviour.