Page MenuHomePhabricator

"HCaptchaBlockedIpEditingScoreCollectionSiteKey" poisons pageview cache (5% latency regression on MediaWiki Backend Pageview Timing)
Closed, ResolvedPublicBUG REPORT

Description

5% ~20ms : ConfirmEdit (HCaptcha) extension Hooks::onMakeGlobalVariablesScript is calling User::getBlock (via addIPBlocksScoreCollectionVars) on every pageview. This is a regression introduced last month in T426943. It adds 5% to all latencies through multiple network roundtrips (new connection to GlobalBlocking DB, DB queries, memcached queries). At glance, this doesn't actually work because logged-out pageviews are CDN-cached so this is either an expensive no-op or actively poisoning caches. I've confirmed just now that when perfoming a logged-out pageview from a VPN connection (one that we block from editing) my cache miss poisoins the CDN cache with HCaptchaBlockedIpEditingScoreCollectionSiteKey.

Steps to reproduce:

  • Request https://en.wikipedia.org/wiki/Banana in private browsing, and view source to confirm wgHCaptchaBlockedIpEditingScoreCollectionSiteKey is not there for you (assuming your home IP is not blocked).
  • Enable a VPN of which the IP range is globally blocked on Meta-Wiki, e.g. Proton VPN.
  • Request https://en.wikipedia.org/wiki/Banana?_cachebust=foobar in private browsing, and view source to confirm wgHCaptchaBlockedIpEditingScoreCollectionSiteKey is now there. Also note that the X-Cache-Status header shows a cache miss, not a cache pass.
  • Hard refresh, and confirm this is a CDN cache hit.

/cc @Dreamy_Jazz @kostajh per https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ConfirmEdit/+/1295484

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Krinkle updated the task description. (Show Details)
Krinkle added subscribers: kostajh, Dreamy_Jazz.

You can verify any fix after rollout via MediaWiki Entrypoint Profiling: One component under index_view/ext_ConfirmEdit:

Screenshot 2026-07-20 at 15.26.02.png (2,314×846 px, 165 KB)

You can verify any fix after rollout via MediaWiki Entrypoint Profiling: One component under index_view/ext_ConfirmEdit:

Screenshot 2026-07-20 at 15.26.02.png (2,314×846 px, 165 KB)

Looks better now after wmf.12 has rolled out

image.png (3,158×820 px, 180 KB)

From the Grafana: backend-pageview-timing dashboard:

  • The p75 quantile is about 10% or 20ms lower than the same time last week (240ms → 220ms).
  • About 5% of traffic moved from the >250ms bucket to <=250ms.
  • And another 5% moved from >100ms to <=100ms.

Screenshot 2026-07-24 at 20.06.58.png (2,087×524 px, 145 KB)

Screenshot 2026-07-24 at 19.26.48.png (1,027×836 px, 134 KB)

Screenshot 2026-07-24 at 19.16.39.png (1,011×831 px, 77 KB) Screenshot 2026-07-24 at 19.17.29.png (980×825 px, 78 KB)

Note that a similar size performance improvement from T374761 rolled out with the same train.

Krinkle changed the visibility from "Custom Policy" to "Public (No Login Required)".Jul 24 2026, 6:31 PM
Krinkle changed the edit policy from "Custom Policy" to "All Users".
Krinkle changed the subtype of this task from "Security Issue" to "Bug Report".

This appears to be fixed:

image.png (1,880×831 px, 155 KB)