Page MenuHomePhabricator

Deploy Temporary accounts to all remaining small-sized projects
Closed, ResolvedPublic

Assigned To
Authored By
Niharika
Aug 18 2025, 11:07 AM
Referenced Files
F65949476: image.png
Sep 2 2025, 6:05 PM
F65949317: image.png
Sep 2 2025, 5:27 PM
F65949377: image.png
Sep 2 2025, 5:27 PM
F65949308: image.png
Sep 2 2025, 5:27 PM
Restricted File
Sep 2 2025, 3:40 PM
F65948862: image.png
Sep 2 2025, 3:40 PM
F65948860: image.png
Sep 2 2025, 3:40 PM

Description

This task is to enable Temporary accounts on all except the following wikis:

bnwiki
cawiki
commonswiki
elwiki
enwiki
enwikisource
enwiktionary
eswiki
fiwiki
huwiki
itwiki
ruwiki
simplewiki
svwiki
thwiki
uzwiki
wikidatawiki
ptwikibooks
svwikisource
enwikinews

Deployment date: September 2, 2025

Event Timeline

Thinking through how to do this, I think we should add all the wikis that are currently in group 0 to tempaccounts.dblist (the list of wikis with temporary accounts enabled), rather than enforcing that temp accounts are enabled on all wikis listed in group0.dblist (the list of group 0 wikis).

Two side effects that could occur if we enable temp accounts on all wikis in group0.dblist:

  1. If a wiki is moved into group 0, they will automatically get temporary accounts enabled, which may not be desired. (This happens occasionally, e.g. https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1076451 .)
  2. If a wiki is moved out of group 0, they will automatically get temporary accounts disabled again. This would be bad, because IPs would start showing up again, and the wiki wouldn't recognize any existing temporary accounts as temporary accounts. (I didn't actually find an example of a wiki being moved out of group 0.)

If we do copy over the group0.dblist wikis to tempaccounts.dblist, we won't be able to assume that all group 0 wikis have temp accounts enabled, so as we plan future rollouts, we may need to check this again. (E.g. if the next rollout wants temporary accounts enabled on all group1 and group0 wikis, we'll need to check the group 0 wikis list again.) But this shouldn't be too cumbersome since rollout should be fairly quick, and group 0 membership doesn't actually change often.

Thinking through how to do this, I think we should add all the wikis that are currently in group 0 to tempaccounts.dblist (the list of wikis with temporary accounts enabled), rather than enforcing that temp accounts are enabled on all wikis listed in group0.dblist (the list of group 0 wikis).

+1 to this solution. We want the list to be fixed to avoid any chance that a wiki could move from "enabled" to "disabled without known". The rollout should be fast enough now that the chance we see a mismatch should be low to none, and even if it does occur it won't be long before they are included in the rollout.

Change #1180532 had a related patch set uploaded (by Tchanders; author: Tchanders):

[operations/mediawiki-config@master] Enable temporary accounts on remaining small-sized projects

https://gerrit.wikimedia.org/r/1180532

The patch at https://gerrit.wikimedia.org/r/1180532 adds all the wikis defined in the task description to the list of wikis with temporary accounts enabled.

The patch is quite difficult to review, as it is almost all wikis.

We could instead define a list of wikis that should not have temporary accounts enabled. That list would now be quite short:

aawiki
annwiki
bclwikisource
bdrwiki
bnwiki
cawiki
commonswiki
elwiki
enwiki
enwikisource
enwiktionary
eswiki
fiwiki
gorwikiquote
huwiki
ibawiki
idwikivoyage
itwiki
kawikisource
kgewiki
kncwiki
madwikisource
madwiktionary
minwikibooks
moswiki
mywikisource
nrwiki
nupwiki
rkiwiki
rskwiki
ruwiki
satwiktionary
shnwikinews
simplewiki
svwiki
sylwiki
tcywikisource
tcywiktionary
tddwiki
thwiki
tigwiki
tlwikisource
uzwiki
wikidatawiki
zghwiktionary

@Niharika @sgrabarczuk Would it be acceptable to define in our configuration that all wikis have temporary accounts by default, unless they are included in this list?

This should also solve the problem raised by @sgrabarczuk about enabling temporary accounts on Wikimedia Incubator, where we want to avoid a scenario where temporary accounts are on Incubator but are not on new wikis. Any new wiki being added would automatically have temporary accounts enabled, unless explicitly added to the list above.

Feedback from @sgrabarczuk is that aawiki should not be on the temp-accounts-disabled list (it's closed), nor should some of the small, non-English projects.

I'll make a patch that uses this exclusion list, and we can update that patch before deployment, once we know the final list for sure.

cc @STran, @Niharika

Feedback from @sgrabarczuk is that aawiki should not be on the temp-accounts-disabled list (it's closed), nor should some of the small, non-English projects.

I'll make a patch that uses this exclusion list, and we can update that patch before deployment, once we know the final list for sure.

cc @STran, @Niharika

The proposed update from @sgrabarczuk is to add the wikis listed in P81430#327326 to the list of wikis to have temp accounts enabled in this deployment.

Then the remaining wikis without temporary accounts after this deployment would be these:

bnwiki
cawiki
commonswiki
elwiki
enwiki
enwikisource
enwiktionary
eswiki
fiwiki
huwiki
itwiki
ruwiki
simplewiki
svwiki
thwiki
uzwiki
wikidatawiki

cc @Quiddity

@Tchanders @STran we want to add ptwikibooks to the exclusion list because their community is still a heavy user of Flow and LqT.

See this thread for more context.

@Tchanders @STran We also want to remove svwikisource and enwikinews for the same reason.

kostajh set Due Date to Sep 1 2025, 10:00 PM.

Change #1180532 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable temporary accounts on remaining small-sized projects

https://gerrit.wikimedia.org/r/1180532

Mentioned in SAL (#wikimedia-operations) [2025-09-02T13:04:58Z] <stran@deploy1003> Started scap sync-world: Backport for [[gerrit:1154308|Document that IP reveal permissions can't just be reassigned (T396217)]], [[gerrit:1180532|Enable temporary accounts on remaining small-sized projects (T402181)]]

Mentioned in SAL (#wikimedia-operations) [2025-09-02T13:11:47Z] <stran@deploy1003> tchanders, stran: Backport for [[gerrit:1154308|Document that IP reveal permissions can't just be reassigned (T396217)]], [[gerrit:1180532|Enable temporary accounts on remaining small-sized projects (T402181)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Niharika awarded a token.
jijiki reopened this task as In Progress.Sep 2 2025, 3:40 PM
jijiki raised the priority of this task from High to Unbreak Now!.
jijiki subscribed.

Hey folks, after the deployment at ~13:15 UTC, I observed the following

  • Increase in memcached traffic

~40% in write traffic

image.png (548×1 px, 97 KB)

~30% in read traffic

image.png (534×1 px, 100 KB)

{F65948996}

I am guessing the above have lead to a mw p50 latency increase, especially mw-web Mediawiki vs the Databse

The relevant keygroups following the pattern are SqlBlobStore_blob, revision_row_1_29, page
https://grafana.wikimedia.org/goto/jclfNH9HR?orgId=1

If there are no external factors causing this, we should revert this change

jijiki lowered the priority of this task from Unbreak Now! to High.Sep 2 2025, 4:04 PM

After discussing with @Dreamy_Jazz @kostajh and @Ladsgroup, we have doubts it the above observations are caused by the change, so I am lowering the priority while we investigate.

The overall impact is some increased latency on mediawiki. Please close this if it turns out the deployment was not related, and it was just unfortunate timing

Some notable jumps that line up exactly with these increases are a significant jump in mw-web 200s:

image.png (1×2 px, 147 KB)

Notably this does not seem to coincide with a significant increase in external requests that we can easily discern

Digging into this we can also see a shift in requests that were previously emitting 304s becoming 200s (which in turn would lead to increased load and page generation in theory)

image.png (602×1 px, 72 KB)

Possibly a side-effect: A notable increase in parsoidCachePrewarm jobs in both datacentres:

image.png (457×2 px, 91 KB)

Following up, as @hnowlan notes, there's a clear inversion between 304 and 200 responses from mw-web starting shortly after 13:10. Further, we don't see a clear source of new additional external traffic that correlates.

However, if we specifically focus on cache misses at the CDN, we can see a clear correlated increase, seemingly entirely focused on enwiki.

So, something changed in enwiki, causing a visible reduction in CDN cache hit rate (change to a common template?). This too would be consistent with the influx of parsoidCachePrewarm jobs.

Unless there's something very subtle here that I'm missing, given the limited scope of wikis to which https://gerrit.wikimedia.org/r/1180532 is applicable (which does not include enwiki), I'm fairly confident the temporal correlation is an unfortunate coincidence.

change to a common template?

image.png (83×694 px, 12 KB)

Which correlates to both the first bump and its little bump on top.

Thank you for pointing that out, @Izno - yes, that correlates quite well and looks like a solid candidate to explain this.

Given the above, I'm going to boldly reclose this task.

Change #1185838 had a related patch set uploaded (by STran; author: STran):

[operations/mediawiki-config@master] Enable temporary accounts on metawiki

https://gerrit.wikimedia.org/r/1185838

Change #1185838 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable temporary accounts on metawiki

https://gerrit.wikimedia.org/r/1185838

Mentioned in SAL (#wikimedia-operations) [2025-09-10T12:46:28Z] <tchanders@deploy1003> Started scap sync-world: Backport for [[gerrit:1185845|Enable temporary accounts on all medium-sized projects (T403399)]], [[gerrit:1185838|Enable temporary accounts on metawiki (T402181)]]

Mentioned in SAL (#wikimedia-operations) [2025-09-10T12:50:52Z] <tchanders@deploy1003> tchanders, stran: Backport for [[gerrit:1185845|Enable temporary accounts on all medium-sized projects (T403399)]], [[gerrit:1185838|Enable temporary accounts on metawiki (T402181)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2025-09-10T13:02:35Z] <tchanders@deploy1003> Finished scap sync-world: Backport for [[gerrit:1185845|Enable temporary accounts on all medium-sized projects (T403399)]], [[gerrit:1185838|Enable temporary accounts on metawiki (T402181)]] (duration: 16m 06s)