en.wiki slow to respond when editing, and occasionally throws an error with Chrome search shortcuts, or blocked because missing HTTPS
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	MBinder_WMF
	Jan 8 2023, 7:55 AM

Description

These might be 3 separate issues. I'm reporting them together because this behavior is all happening within the last 6 hours or so.

en.wiki slow to respond

When editing any page, after using Visual Editor templates for citation and/or links, the page slows to a crawl, and simple actions like typing take several seconds to respond.

occasionally throws an error with Chrome search shortcuts

Using Chrome Version 108.0.5359.124 (Official Build) (arm64) and site search shortcuts (in this case, "w" and "http://en.wikipedia.org/wiki/Special:Search?search=%s"), sometimes the site will throw a one-line error. It's unpredictable and lingers only for a few seconds, so I haven't been able to capture or replicate consistently.

blocked because missing HTTPS

When trying to visit wikipedia.org, where normally I get an instant and direct connection to the portal, Chrome says the site does not use HTTPS. After a moment, or a hard refresh, it resolves. This happens on other trusted HTTPS sites from time to time, too, so normally I wouldn't think twice, but I'm listing it here given the other performance issues I've encountered.

Environment: macOS 12.6 on a WMF-issued 16-inch M1 Max

Related Objects

Duplicates Merged Here: T326495: 2023-01-08 Wikimedia Connectivity Issues
T326497: HTTP 503 and 504 errors in Mon Wiktionary

Event Timeline

MBinder_WMF created this task.Jan 8 2023, 7:55 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 8 2023, 7:55 AM

Have these issues been happening for a while or only recently in the last hour or so, WikiMedia Status does show a spike in errors about half an hour ago

There’s a marked spike in errors and a flurry of icinga alerts. Can someone please look?

RhinosF1 added a project: Wikimedia-Incident.Jan 8 2023, 8:56 AM

07:37:20 <icinga-wm> PROBLEM - BGP status on cr2-eqsin is CRITICAL: BGP CRITICAL - No response from remote host 103.102.166.130 https://wikitech.wikimedia.org/wiki/Network_monitoring%23BGP_status
07:37:46 <icinga-wm> PROBLEM - BGP status on cr3-eqsin is CRITICAL: BGP CRITICAL - No response from remote host 103.102.166.131 https://wikitech.wikimedia.org/wiki/Network_monitoring%23BGP_status
07:38:24 <icinga-wm> PROBLEM - Host cp5024 is DOWN: PING CRITICAL - Packet loss = 100%

(and more hosts + mgmt)

07:39:18 <jinxer-wm> (ProbeDown) firing: Service text-https:443 has failed probes (http_text-https_ip6) #page - https://wikitech.wikimedia.org/wiki/Runbook#text-https:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/service&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown
(LOTS OF FLAPPING ALERTS, Pybal, hosts down, DNS errors)
07:50:20 <jinxer-wm> (ProbeDown) resolved: (5) Service text-https:443 has failed probes (http_text-https_ip4) #page - https://wikitech.wikimedia.org/wiki/Runbook#text-https:443 - https://grafana.wikimedia.org/d/O0nHhdhnz/network-probes-overview?var-job=probes/service&var-module=All - https://alerts.wikimedia.org/?q=alertname%3DProbeDown

I can’t see any actual page firing in klaxon and no public response from SRE yet.

RhinosF1 added a project: Traffic.Jan 8 2023, 9:01 AM

This task reports continuing issues for hours, is unrelated to the issue that happened about 1 hour ago and that was handled by SRE

I can report today that while the site is still slower than I'd expect, there was a marked improvement. OK to close this for now and re-open, as needed (or as I can replicate better).

larissagaulia closed this task as Resolved.Jan 12 2023, 7:08 AM

larissagaulia claimed this task.

Stang unsubscribed.Jan 15 2023, 10:33 PM

Ericliu1912 unsubscribed.Mar 6 2023, 9:08 AM

en.wiki slow to respond when editing, and occasionally throws an error with Chrome search shortcuts, or blocked because missing HTTPSClosed, ResolvedPublicActions