Page MenuHomePhabricator

COIBot repeatedly getting logged out of projects
Open, Needs TriagePublic

Description

Will raise this directly with @Beetstra, but just logging here as well (don't think there's a more specific project to use than VPS-Projects?)

IRC logs in #wikimedia-external-links show this has been going on since around Wednesday, December 21st, 2022 — logs below from some point during the 22nd:

10:35:48 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
10:35:54 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
10:36:05 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
10:36:17 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
10:36:28 <+COIBot> Got logged out from en.wikipedia.org - trying to log in as COIBot.
10:36:44 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
10:36:54 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
10:37:05 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
10:37:16 <+COIBot> Got logged out from en.wikipedia.org - trying to log in as COIBot.
10:37:28 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:10:41 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:10:48 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:10:59 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:11:11 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:11:22 <+COIBot> Got logged out from en.wikipedia.org - trying to log in as COIBot.
11:15:10 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:15:17 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:15:28 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:15:40 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:15:51 <+COIBot> Got logged out from en.wikipedia.org - trying to log in as COIBot.
11:16:14 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:16:22 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:16:34 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:16:45 <+COIBot> Got logged out from en.wikipedia.org - trying to log in as COIBot.
11:16:56 <+COIBot> Got logged out from en.wikipedia.org - trying to log in as COIBot.
11:17:07 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:17:24 <+COIBot> Saved an XWiki report for techmotech.com (XWiki spam by Rikubarik).
11:18:42 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:18:49 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:19:00 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:19:12 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.
11:19:23 <+COIBot> Got logged out from en.wikipedia.org - trying to log in as COIBot.
11:26:53 <+COIBot> Got logged out from meta.wikimedia.org - trying to log in as COIBot.

Event Timeline

Noting this comment on Meta by @Billinghurst:

COIBot was logged out and chiming in on meta that it was the case. A soft reboot of the instance just seem to have been totally ineffective as there is now no IRC either. I still haven't figured out my bastion access since they killed the access via the tools bastion.

Was resolved prior to the recent T326014: coibot.linkwatcher out of disk space, after which it failed again — seems to be working now...

Mentioned in SAL (#wikimedia-cloud) [2022-12-31T04:44:34Z] <TheresNoTime> restarted coibot/refreshed login - T325867