The homepage works: https://outreachdashboard.wmflabs.org/. But most other things seem to just hang forever, for example: trying to login, clicking "Find Programs", course URLs such as https://outreachdashboard.wmflabs.org/courses/Les_sans_pagEs/Violette_and_Co_2025_(samedi_25_janvier_2025), and probably more.
Description
Description
Related Objects
Related Objects
- Mentioned Here
- T383238: [nfs] 2025-01-08 tools-nfs outage
Event Timeline
Comment Actions
May be related to T383238: [nfs] 2025-01-08 tools-nfs outage? (I don’t know to what extent those URLs would rely on NFS.)
Comment Actions
Yes, I think it was related. It was hanging on fetching two Vega libraries that are hosted on toolforge (instead of a commercial CDN). I stopped and started the webservice for wikiedudashboard tool, and that fixed it.
Comment Actions
@Samwalton9-WMF this time, it was good old-fashioned unresponsive server (so the homepage also was not loading). I believe this happens when the combination of web requests and the background workers use up all the database compute; without intervention, it typically comes back online within an hour, and restarting the peony-web instance via Horizon fixes it quickly.