SUL3 is on the testwikis now, and could use some extra manual testing.
Some things that could be tested:
== Login ==
[X] basic password login - PASSED ✅
[X] "keep me logged in checkbox" (should result in `centralauth_Token` cookie with 1-year expiry on the wiki where you are logging in) - PASSED ✅
NOTE: [Test Above] The `centralauth_Token` cookie is set on the top-level/parent domain `.wikipedia.org` and has a 1 year expiry (when I log in via test wikipedia) and also the 1-year expiry applies to the local domain where I'm logged into (test.wikipedia.org).
[] forced password change on login when having a weak password
[x] login with a temporary password - PASSED ✅
[x] login with a temporary password, on a different wiki than where the email was sent from - PASSED ✅
NOTE: [Test Above] The temporary password I got on testwiki also worked during login on testwikidata.
[X] captcha appears after a few failed login attempts for the same user, and prevents further login attempts unless correctly filled out - PASSED ✅
NOTE: The above test passes. A CAPTCHA security check is triggered after 3 failed login attempts for the same user.
[x] login gets throttled after even more failed attempts - PASSED ✅
NOTE: [Test Above] After like 7 - 10 times of failed attempt, I got the message `You have made too many recent login attempts. Please wait 5 minutes before trying again.`.
[] blocked user cannot login, gets reasonable error message
[] same for locally spam-blacklisted user (test both the JS dropdown on the username field, and the form submit)
[] same for locally title-blacklisted user (test both the JS dropdown on the username field, and the form submit)
[] same for username blocked by AbuseFilter (test both the JS dropdown on the username field, and the form submit)
[] same when trying to log in on a closed wiki
[] test one of the mitigations in PrivateSettings
[X] TOTP second factor - PASSED ✅
[] WebAuthn second factor (nice to have - T376021; probably no way to set up a working key on auth.wikimedia.org at this point)
[X] starting signup, switching to login via user menu - PASSED ✅
[] security reauthentication (e.g. when doing a password change) works
[] OAuth flow: use OAuth-based identity while not being logged in on Wikimedia (the OAuth Authorization would have to happen on testwiki, not sure if there's an existing tool like that, or we need to create a new tool for testing)
[x] login via a permission error redirect (e.g. visit Special:Preferences while logged out) - PASSED ✅
NOTE: [Test Above] Visited `Special:Preferences` and got redirected to the shared domain login page. After successful login, get redirected back to the special preferences page.
[x] NewUserMessage autocreate welcome message - PASSED ✅
NOTE: The links are correct too: https://test2.wikipedia.org/wiki/User_talk:Tgr-test-c1121055
[x] checkuser data is logged after successful login, including client hints - PASSED ✅
[x] checkuser data is logged after failed login login, including client hints - PASSED ✅
[x] LoginNotify email is sent after login from new device - PASSED ✅
[x] LoginNotify email is sent after failed login - PASSED ✅
[] login via fallback URL (once T377140 is done)
== Signup ==
[x] basic user account creation - PASSED ✅
[x] email notification gets sent, links use canonical domain - PASSED ✅
NOTE: [Test Above] Yes, the link works correctly. It's in the canonical form e.g. https://test.wikipedia.org/wiki/Special:ConfirmEmail/<code-REDACTED>
[x] captcha works - PASSED ✅
[] signup gets throttled after a few successful signups from the same IP; can be unthrottled with [[https://wikitech.wikimedia.org/wiki/Increasing_account_creation_threshold | resetAuthenticationThrottle]]
[x] blocked user cannot sign up, gets reasonable error message - PASSED ✅
[] same for spam-blacklisted user (test both the JS dropdown on the username field, and the form submit)
[] same for title-blacklisted user (test both the JS dropdown on the username field, and the form submit)
[] same for username caught by AntiSpoof (test both the JS dropdown on the username field, and the form submit)
[] same for username blocked by AbuseFilter (test both the JS dropdown on the username field, and the form submit)
[] signup is disallowed on a closed wiki
[] something something IPReputation? not sure if this is testable in production
[x] starting login, switching to signup via form button - PASSED ✅
NOTE: [Test Above] On testwiki, the link that this button sits on has a `campaign=loginCTA` parameter in the URL and that works correctly too.
[x] starting login, switching to signup via user menu - PASSED ✅
[x] GrowthExperiments signup flow (signup should end with Special:WelcomeSurvey) - PASSED ✅
[] GrowthExperiments signup-during-edit flow (signup from VE should end with Special:WelcomeSurvey after page save)
[x] [[https://www.mediawiki.org/wiki/Extension:GrowthExperiments/Technical_documentation/Campaigns/Creation_of_customized_landing_pages|customized landing pages]] (especially that it doesn't result in auth.wikimedia.org URLs for some inolved page / message due to parser cache pollution) - PASSED ✅
NOTE: Per [[https://gerrit.wikimedia.org/g/operations/mediawiki-config/+/1cfe88d881f22c1a9807f1f95c367ea1dbd627db/wmf-config/ext-GrowthExperiments.php#768|GE campaign pattern]], I tested with https://test.wikipedia.org/w/index.php?title=Special:CreateAccount&returnto=Main+Page&campaign=typage-test-test-1, you can see that on the right side of the signup form, it looks different from without that campaign param supplied.
[X] NewUserMessage welcome message (especially that it doesn't end up with auth.wikimedia.org URLs due to parser cache pollution) - PASSED ✅
[X] `campaign` URL parameter results in user preference correctly set - PASSED ✅
[] `incubatortestwiki-project`/`incubatortestwiki-code` user preferences correctly set when using a signup link with `testwikiproject` / `testwikicode` query parameters on Incubator - FAILED ❌
NOTE: [Test above] Wikimedia Incubator extension is not enabled on test wikis it seems.
[x] temp user creation via edit works in basic editor - PASSED ✅
[x] temp user creation via edit works in some JS editor, e.g. DiscussionTools - PASSED ✅
NOTE: [Test Above] The discussion topic created (with a corresponding temp account) that is signed by the temp account. See: https://test.wikipedia.org/w/index.php?title=Talk%3ATest_page#Test_test
[X] temp user signing up for named account - PASSED ✅
== API ==
[X] login via action=clientlogin (on a local domain) - PASSED ✅
[x] bot login via action=login - PASSED ✅
[X] action=logout - PASSED ✅
(future: make sure credentials change APIs are unaffected after T362715)
== Central session ==
[X] after login or signup, user should be logged in on other registrable domains - PASSED ✅
NOTE: [Test Above] Logged into test.wikipedia.org and was also logged into test2.wikipedia.org. Also visited test.wikidata.org, it detected that I was centrally logged in and I reloaded to gain a session there.
[x] "keep me logged in" state is transferred correctly - PASSED ✅
[x] after deleting cookies on a given domain, centrally logged-in user should autologin - PASSED ✅
[x] after deleting cookies on a given domain and setting `CentralAuthAnon=1` cookie, centrally logged-in user should autologin when clicking login link - PASSED ✅
NOTE: [Test Above] Cleared all cookies on the given domain and set only the `CentralAuthAnon` cookie to value `1` and it works.
[] after temp user creation, temp user should be logged in on other registrable domains - FLAKY 🚧
NOTE: [Test Above] This doesn't work as expected. If I make an edit as anon user on test.wikipedia.org, I was expecting that going to test2.wikipedia.org, I would be logged-in there but I don't have that experience. Visiting test2wiki shows I logged out (no temp account). I filed {T383136} related to it. To add, this is flaky, doesn't work all the time (works sometimes <~2025-24242 on https://test.wikipedia.org/wiki/Test_page> and not for others <~2025-23466 on https://test.wikipedia.org/wiki/Sandbox>), this is the first time I'm experiencing this in production and have experienced this locally too.
[] "keep me logged in" state is transferred correctly - FLAKY 🚧
NOTE: [Test above] Works but flaky as mentioned in the bullet above this one. So when it works, state is transferred but when it doesn't work, it's not transferred.
[] after deleting cookies on a given domain, centrally logged-in temp user should autologin
[] after deleting cookies on a given domain and setting `CentralAuthAnon=1` cookie, centrally logged-in temp user should autologin when clicking login link
[x] logout clears the `centralauth_*` cookies on registrable domains other than the current one - PASSED ✅
== Instrumentation ==
NOTE: followed the docs at https://wikitech.wikimedia.org/wiki/Event_Platform/Instrumentation_How_To#EventStreams and it seems to work nicely.
[x] [[https://schema.wikimedia.org/repositories//secondary/jsonschema/analytics/mediawiki/accountcreation/account_conversion/current.yaml|accountcreation/account_conversion]] gets logged during login page view + after successful login, and has correct SUL3 flag - PASSED ✅
[x] [[https://schema.wikimedia.org/repositories/secondary/jsonschema/analytics/legacy/serversideaccountcreation/current.yaml|serversideaccountcreation]] gets logged after signup and has correct SUL3 flag - PASSED ✅
[x] [[https://schema.wikimedia.org/repositories//secondary/jsonschema/analytics/mediawiki/accountcreation/account_conversion/current.yaml|accountcreation/account_conversion]] gets logged during signup page view + after successful signup, and has correct SUL3 flag - PASSED ✅
[x] [[https://schema.wikimedia.org/repositories//secondary/jsonschema/analytics/mediawiki/accountcreation/block/current.yaml|accountcreation/block]] gets logged after signup attempt from a blocked IP, and has correct SUL3 flag - PASSED ✅
[] `sul3_authentication_start_total` / `sul3_authentication_end_total` gets incremented during login / signup
== Rollout ==
[] using the fallback login link results in SUL2 login workflow. Central autologin remains functional. (once T377140 is done)
[] same for signup. User still gets opted into SUL3.
[] when signing up on a SUL3-for-signup-enabled wiki, the next login on the same registrable domain will result in a SUL3 flow. (once T377144 is done)
[] when an existing user got opted into SUL3, their next login (on any registrable domain where they have logged in recently) will result in a SUL3 flow. (once T384215 is done)
== Other ==
[] taking a long time to fill out the login form (more than the 5 minute login session expiry)
NOTE: this behaves poorly in the scenario where SUL3 is enabled for signup but not login: user goes to shared domain, creates account after 5+ minutes, ends up on the local domain login page with a session expiry error. The signup was successful but 1) no way to know, 2) it's not clear to the user why they are seeing a login form when they were trying to sign up.
(future: credentials change workflows after T362715)