| Niharika | |
| Aug 18 2025, 11:07 AM |
| F65949476: image.png | |
| Sep 2 2025, 6:05 PM |
| F65949317: image.png | |
| Sep 2 2025, 5:27 PM |
| F65949377: image.png | |
| Sep 2 2025, 5:27 PM |
| F65949308: image.png | |
| Sep 2 2025, 5:27 PM |
| Restricted File | |
| Sep 2 2025, 3:40 PM |
| F65948862: image.png | |
| Sep 2 2025, 3:40 PM |
| F65948860: image.png | |
| Sep 2 2025, 3:40 PM |
This task is to enable Temporary accounts on all except the following wikis:
bnwiki cawiki commonswiki elwiki enwiki enwikisource enwiktionary eswiki fiwiki huwiki itwiki ruwiki simplewiki svwiki thwiki uzwiki wikidatawiki ptwikibooks svwikisource enwikinews
Deployment date: September 2, 2025
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Restricted Task | |||||
| Resolved | kostajh | T294511 2021 Security Team wikireplicas audit | |||
| Declined | None | T284948 Raw IPs of logged-out users disclosed in wiki-replicas | |||
| Resolved | Niharika | T324492 Temporary accounts - MVP | |||
| Open | None | T340001 [Epic] Deployment plan for Temporary Accounts | |||
| Resolved | Tchanders | T402181 Deploy Temporary accounts to all remaining small-sized projects |
Thinking through how to do this, I think we should add all the wikis that are currently in group 0 to tempaccounts.dblist (the list of wikis with temporary accounts enabled), rather than enforcing that temp accounts are enabled on all wikis listed in group0.dblist (the list of group 0 wikis).
Two side effects that could occur if we enable temp accounts on all wikis in group0.dblist:
If we do copy over the group0.dblist wikis to tempaccounts.dblist, we won't be able to assume that all group 0 wikis have temp accounts enabled, so as we plan future rollouts, we may need to check this again. (E.g. if the next rollout wants temporary accounts enabled on all group1 and group0 wikis, we'll need to check the group 0 wikis list again.) But this shouldn't be too cumbersome since rollout should be fairly quick, and group 0 membership doesn't actually change often.
+1 to this solution. We want the list to be fixed to avoid any chance that a wiki could move from "enabled" to "disabled without known". The rollout should be fast enough now that the chance we see a mismatch should be low to none, and even if it does occur it won't be long before they are included in the rollout.
Change #1180532 had a related patch set uploaded (by Tchanders; author: Tchanders):
[operations/mediawiki-config@master] Enable temporary accounts on remaining small-sized projects
The patch at https://gerrit.wikimedia.org/r/1180532 adds all the wikis defined in the task description to the list of wikis with temporary accounts enabled.
The patch is quite difficult to review, as it is almost all wikis.
We could instead define a list of wikis that should not have temporary accounts enabled. That list would now be quite short:
aawiki annwiki bclwikisource bdrwiki bnwiki cawiki commonswiki elwiki enwiki enwikisource enwiktionary eswiki fiwiki gorwikiquote huwiki ibawiki idwikivoyage itwiki kawikisource kgewiki kncwiki madwikisource madwiktionary minwikibooks moswiki mywikisource nrwiki nupwiki rkiwiki rskwiki ruwiki satwiktionary shnwikinews simplewiki svwiki sylwiki tcywikisource tcywiktionary tddwiki thwiki tigwiki tlwikisource uzwiki wikidatawiki zghwiktionary
@Niharika @sgrabarczuk Would it be acceptable to define in our configuration that all wikis have temporary accounts by default, unless they are included in this list?
This should also solve the problem raised by @sgrabarczuk about enabling temporary accounts on Wikimedia Incubator, where we want to avoid a scenario where temporary accounts are on Incubator but are not on new wikis. Any new wiki being added would automatically have temporary accounts enabled, unless explicitly added to the list above.
Feedback from @sgrabarczuk is that aawiki should not be on the temp-accounts-disabled list (it's closed), nor should some of the small, non-English projects.
I'll make a patch that uses this exclusion list, and we can update that patch before deployment, once we know the final list for sure.
The proposed update from @sgrabarczuk is to add the wikis listed in P81430#327326 to the list of wikis to have temp accounts enabled in this deployment.
Then the remaining wikis without temporary accounts after this deployment would be these:
bnwiki cawiki commonswiki elwiki enwiki enwikisource enwiktionary eswiki fiwiki huwiki itwiki ruwiki simplewiki svwiki thwiki uzwiki wikidatawiki
cc @Quiddity
@Tchanders @STran we want to add ptwikibooks to the exclusion list because their community is still a heavy user of Flow and LqT.
See this thread for more context.
@Tchanders @STran We also want to remove svwikisource and enwikinews for the same reason.
Change #1180532 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable temporary accounts on remaining small-sized projects
Mentioned in SAL (#wikimedia-operations) [2025-09-02T13:04:58Z] <stran@deploy1003> Started scap sync-world: Backport for [[gerrit:1154308|Document that IP reveal permissions can't just be reassigned (T396217)]], [[gerrit:1180532|Enable temporary accounts on remaining small-sized projects (T402181)]]
Mentioned in SAL (#wikimedia-operations) [2025-09-02T13:11:47Z] <stran@deploy1003> tchanders, stran: Backport for [[gerrit:1154308|Document that IP reveal permissions can't just be reassigned (T396217)]], [[gerrit:1180532|Enable temporary accounts on remaining small-sized projects (T402181)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
Hey folks, after the deployment at ~13:15 UTC, I observed the following
~40% in write traffic
~30% in read traffic
{F65948996}
I am guessing the above have lead to a mw p50 latency increase, especially mw-web Mediawiki vs the Databse
The relevant keygroups following the pattern are SqlBlobStore_blob, revision_row_1_29, page
https://grafana.wikimedia.org/goto/jclfNH9HR?orgId=1
If there are no external factors causing this, we should revert this change
After discussing with @Dreamy_Jazz @kostajh and @Ladsgroup, we have doubts it the above observations are caused by the change, so I am lowering the priority while we investigate.
The overall impact is some increased latency on mediawiki. Please close this if it turns out the deployment was not related, and it was just unfortunate timing
Some notable jumps that line up exactly with these increases are a significant jump in mw-web 200s:
Digging into this we can also see a shift in requests that were previously emitting 304s becoming 200s (which in turn would lead to increased load and page generation in theory)
Possibly a side-effect: A notable increase in parsoidCachePrewarm jobs in both datacentres:
Following up, as @hnowlan notes, there's a clear inversion between 304 and 200 responses from mw-web starting shortly after 13:10. Further, we don't see a clear source of new additional external traffic that correlates.
However, if we specifically focus on cache misses at the CDN, we can see a clear correlated increase, seemingly entirely focused on enwiki.
So, something changed in enwiki, causing a visible reduction in CDN cache hit rate (change to a common template?). This too would be consistent with the influx of parsoidCachePrewarm jobs.
Unless there's something very subtle here that I'm missing, given the limited scope of wikis to which https://gerrit.wikimedia.org/r/1180532 is applicable (which does not include enwiki), I'm fairly confident the temporal correlation is an unfortunate coincidence.
Thank you for pointing that out, @Izno - yes, that correlates quite well and looks like a solid candidate to explain this.
Change #1185838 had a related patch set uploaded (by STran; author: STran):
[operations/mediawiki-config@master] Enable temporary accounts on metawiki
Change #1185838 merged by jenkins-bot:
[operations/mediawiki-config@master] Enable temporary accounts on metawiki
Mentioned in SAL (#wikimedia-operations) [2025-09-10T12:46:28Z] <tchanders@deploy1003> Started scap sync-world: Backport for [[gerrit:1185845|Enable temporary accounts on all medium-sized projects (T403399)]], [[gerrit:1185838|Enable temporary accounts on metawiki (T402181)]]
Mentioned in SAL (#wikimedia-operations) [2025-09-10T12:50:52Z] <tchanders@deploy1003> tchanders, stran: Backport for [[gerrit:1185845|Enable temporary accounts on all medium-sized projects (T403399)]], [[gerrit:1185838|Enable temporary accounts on metawiki (T402181)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
Mentioned in SAL (#wikimedia-operations) [2025-09-10T13:02:35Z] <tchanders@deploy1003> Finished scap sync-world: Backport for [[gerrit:1185845|Enable temporary accounts on all medium-sized projects (T403399)]], [[gerrit:1185838|Enable temporary accounts on metawiki (T402181)]] (duration: 16m 06s)
