SUL3 is on the testwikis now, and could use some extra manual testing.
Some things that could be tested:
== Login ==
[X] basic password login - works as expected ✅
[X] "keep me logged in checkbox" (should result in `centralauth_Token` cookie with 1-year expiry on the wiki where you are logging in) ✅
NOTE: [Test Above] The `centralauth_Token` cookie is set on the top-level/parent domain `.wikipedia.org` and has a 1 year expiry (when I log in via test wikipedia) and also the 1-year expiry applies to the local domain where I'm logged into (test.wikipedia.org).
[] forced password change on login when having a weak password
[x] login with a temporary password - works as expected ✅
[x] login with a temporary password, on a different wiki than where the email was sent from - works as expected ✅
NOTE: [Test Above] The temporary password I got on testwiki also worked during login on testwikidata.
[X] captcha appears after a few failed login attempts for the same user, and prevents further login attempts unless correctly filled out ✅
NOTE: The above test passes. A CAPTCHA security check is triggered after 3 failed login attempts for the same user.
[x] login gets throttled after even more failed attempts - works as expected ✅
NOTE: [Test Above] After like 7 - 10 times of failed attempt, I got the message `You have made too many recent login attempts. Please wait 5 minutes before trying again.`.
[] blocked user cannot login, gets reasonable error message
[] same for locally spam-blacklisted user (test both the JS dropdown on the username field, and the form submit)
[] same for locally title-blacklisted user (test both the JS dropdown on the username field, and the form submit)
[] same for username blocked by AbuseFilter (test both the JS dropdown on the username field, and the form submit)
[] same when trying to log in on a closed wiki
[] test one of the mitigations in PrivateSettings
[X] TOTP second factor - works as expected ✅
[] WebAuthn second factor (nice to have - T376021; probably no way to set up a working key on auth.wikimedia.org at this point)
[X] starting signup, switching to login via user menu - works as expected ✅
[] security reauthentication (e.g. when doing a password change) works
[] OAuth flow: use OAuth-based identity while not being logged in on Wikimedia (the OAuth Authorization would have to happen on testwiki, not sure if there's an existing tool like that, or we need to create a new tool for testing)
[x] login via a permission error redirect (e.g. visit Special:Preferences while logged out) - works as expected ✅
NOTE: [Test Above] Visited `Special:Preferences` and got redirected to the shared domain login page. After successful login, get redirected back to the special preferences page.
[] NewUserMessage autocreate welcome message - didn't work as expected ❌
NOTE: [Test Above] Didn't work as expected, I didn't get a welcome message: https://test.wikipedia.org/w/index.php?title=User_talk:Sul3MptTestAccount. Have created 3 accounts already but none of them had a welcome message on their user talk page.
[x] checkuser data is logged after successful login, including client hints - works as expected ✅
[x] checkuser data is logged after failed login login, including client hints - works as expected ✅
[x] LoginNotify email is sent after login from new device - works as expected ✅
[x] LoginNotify email is sent after failed login - works as expected ✅
[] login via fallback URL (once T377140 is done)
== Signup ==
[x] basic user account creation - works as expected ✅
[x] email notification gets sent, links use canonical domain - works as expected ✅
NOTE: [Test Above] Yes, the link works correctly. It's in the canonical form e.g. https://test.wikipedia.org/wiki/Special:ConfirmEmail/<code-REDACTED>
[x] captcha works - works as expected ✅
[] signup gets throttled after a few successful signups from the same IP; can be unthrottled with [[https://wikitech.wikimedia.org/wiki/Increasing_account_creation_threshold | resetAuthenticationThrottle]]
[] blocked user cannot sign up, gets reasonable error message
[] same for spam-blacklisted user (test both the JS dropdown on the username field, and the form submit)
[] same for title-blacklisted user (test both the JS dropdown on the username field, and the form submit)
[] same for username caught by AntiSpoof (test both the JS dropdown on the username field, and the form submit)
[] same for username blocked by AbuseFilter (test both the JS dropdown on the username field, and the form submit)
[] signup is disallowed on a closed wiki
[] something something IPReputation? not sure if this is testable in production
[x] starting login, switching to signup via form button - works as expected ✅
NOTE: [Test Above] On testwiki, the link that this button sits on has a `campaign=loginCTA` parameter in the URL and that works correctly too.
[x] starting login, switching to signup via user menu - works as expected ✅
[x] GrowthExperiments signup flow (signup should end with Special:WelcomeSurvey) - works as expected ✅
[] GrowthExperiments signup-during-edit flow (signup from VE should end with Special:WelcomeSurvey after page save)
[] [[https://www.mediawiki.org/wiki/Extension:GrowthExperiments/Technical_documentation/Campaigns/Creation_of_customized_landing_pages|customized landing pages]] (especially that it doesn't result in auth.wikimedia.org URLs for some inolved page / message due to parser cache pollution)
[] NewUserMessage welcome message (especially that it doesn't end up with auth.wikimedia.org URLs due to parser cache pollution)
[X] `campaign` URL parameter results in user preference correctly set - works as expected ✅
[] `incubatortestwiki-project`/`incubatortestwiki-code` user preferences correctly set when using a signup link with `testwikiproject` / `testwikicode` query parameters on Incubator
[x] temp user creation via edit works in basic editor - works as expected ✅
[x] temp user creation via edit works in some JS editor, e.g. DiscussionTools - works as expected ✅
NOTE: [Test Above] The discussion topic created (with a corresponding temp account) that is signed by the temp account. See: https://test.wikipedia.org/w/index.php?title=Talk%3ATest_page#Test_test
[] temp user signing up for named account
== API ==
[X] login via action=clientlogin (on a local domain) - works as expected ✅
[x] bot login via action=login - works as expected ✅
[X] action=logout - works as expected ✅
(future: make sure credentials change APIs are unaffected after T362715)
== Central session ==
[X] after login or signup, user should be logged in on other registrable domains - works as expected ✅
NOTE: [Test Above] Logged into test.wikipedia.org and was also logged into test2.wikipedia.org. Also visited test.wikidata.org, it detected that I was centrally logged in and I reloaded to gain a session there.
[x] "keep me logged in" state is transferred correctly - works as expected ✅
[x] after deleting cookies on a given domain, centrally logged-in user should autologin - works as expected ✅
[x] after deleting cookies on a given domain and setting `CentralAuthAnon=1` cookie, centrally logged-in user should autologin when clicking login link - works as expected ✅
NOTE: [Test Above] Cleared all cookies on the given domain and set only the `CentralAuthAnon` cookie to value `1` and it works.
[] after temp user creation, temp user should be logged in on other registrable domains ❌
NOTE: [Test Above] This doesn't work as expected. If I make an edit as anon user on test.wikipedia.org, I was expecting that going to test2.wikipedia.org, I would be logged-in there but I don't have that experience. Vising test2wiki shows I logged out (no temp account). I filed {T383136} related to it.
[] "keep me logged in" state is transferred correctly
[] after deleting cookies on a given domain, centrally logged-in temp user should autologin
[] after deleting cookies on a given domain and setting `CentralAuthAnon=1` cookie, centrally logged-in temp user should autologin when clicking login link
[x] logout clears the `centralauth_*` cookies on registrable domains other than the current one - works as expected ✅
== Instrumentation ==
[] [[https://schema.wikimedia.org/repositories//secondary/jsonschema/analytics/mediawiki/accountcreation/account_conversion/current.yaml|accountcreation/account_conversion]] gets logged during login page view + after successful login, and has correct SUL3 flag
[] [[https://schema.wikimedia.org/repositories/secondary/jsonschema/analytics/legacy/serversideaccountcreation/current.yaml|serversideaccountcreation]] gets logged after signup and has correct SUL3 flag
[] [[https://schema.wikimedia.org/repositories//secondary/jsonschema/analytics/mediawiki/accountcreation/account_conversion/current.yaml|accountcreation/account_conversion]] gets logged during signup page view + after successful signup, and has correct SUL3 flag
[] [[https://schema.wikimedia.org/repositories//secondary/jsonschema/analytics/mediawiki/accountcreation/block/current.yaml|accountcreation/block]] gets logged after signup attempt from a blocked IP, and has correct SUL3 flag
[] `sul3_authentication_start_total` / `sul3_authentication_end_total` gets incremented during login / signup
== Rollout ==
[] using the fallback login link results in SUL2 login workflow. Central autologin remains functional. (once T377140 is done)
[] same for signup. User still gets opted into SUL3.
[] when signing up on a SUL3-for-signup-enabled wiki, the next login on the same registrable domain will result in a SUL3 flow. (once T377144 is done)
[] when an existing user got opted into SUL3, their next login (on any registrable domain where they have logged in recently) will result in a SUL3 flow. (once T384215 is done)
== Other ==
[] taking a long time to fill out the login form (more than the 5 minute login session expiry)
(future: credentials change workflows after T362715)