https://meta.wikimedia.org/wiki/User-Agent_policy

Policy-required wikitech-l message: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/VMJXVVAQR6S74PR4M5LMW3EAW7VNM7D6/

Both are only supposed to have -nox (the non-graphical one) installed instead of -gtk. The Puppet code pulls in the emacs meta package, and I suspect something changed between -10 and -11 were provisioned which changed which one apt will install by default. I'll send a patch to update that.

See also T279110: Replace PodSecurityPolicy in Toolforge Kubernetes, would be nice to keep Toolforge and PAWS on similar-ish technology stack.

And tasks can either be in the top level "mwbot-rs" project, or in *one* of the subprojects (but not multiple), correct?

AIUI a task can be in the parent project, or *multiple* subprojects, but not both in the parent and a subproject.

This will most likely cause some issues with at least some cloud vps/toolforge users so we probably want to announce this in advance on cloud-announce@.

Anyhow, I suspect the bullseye replica would need to be re-cloned from the replica since it's been so long without being able to catch up on replication.

In T301949#8262117, @Andrew wrote:

@taavi can you catch me up on the state of this task? Are still still mostly blocked by high ceph latency or did your comment at the end of June imply that you're past that?

Ok, sounds good. Mostly I'd been curious about the plan about other dependencies (databases, caches, ...) if it would've been the latter.

Hey. Is the plan to just test that the servers install all the packages properly or are you actually planning to test that reading/editing works properly?

linking this here: https://wikitech.wikimedia.org/wiki/User:Majavah/Loki_notes

Hi. I'll close this as a duplicate of T276961: Support Openstack Swift APIs via the radosgw, which tracks the overall project of providing some object storage service although the exact interfaces it'll be offering aren't yet known.

Hmm. I've checked the Redis and Keepalived (what we use for providing HA for Redis) logs for the timeout on 24 September, nothing strange there. The Redis docs say that by default either the client nor the server have any idle timeout set, nor does our configuration set any limits.

The "delete global account" feature was originally implemented for the purposes of the SUL migration and it hasn't really been used since. I don't think soft deletion is the way to go here.

@Andrew looks like you removed the NFS server while they were still mounted on the existing instances? I fixed that by removing the NFS mounts and mounting the Cinder volume on utrs-production with a symlink on /home.

Ok, all done I think. Congrats!

As far as I can tell this has been open for more than the minimum amount of time (1 week) with multiple supports and zero objections. I'll implement this in the next few days.

These are managed by acme-chief, and I see this in the logs:

Sep 18 15:33:00 tools-acme-chief-01 acme-chief-backend[4225]: Handling pushed CSR event for toolserver / rsa-2048
Sep 18 15:33:02 tools-acme-chief-01 acme-chief-backend[4225]: DNS server 208.80.154.135 (ACMEChallengeValidation.UNKNOWN) failed to validate challenge Challenge type: ACMEChallengeType.DNS01. _acme-challenge.toolserver.org TXT LPo11Ip3SBEw0nuVFz4oAsXKPo4mbfVrmFvCwwie1eA
Sep 18 15:33:02 tools-acme-chief-01 acme-chief-backend[4225]: Unable to validate challenge Challenge type: ACMEChallengeType.DNS01. _acme-challenge.toolserver.org TXT LPo11Ip3SBEw0nuVFz4oAsXKPo4mbfVrmFvCwwie1eA

Hello again. Your user account is again showing up in the list of largest disk space users with roughly 200G of data in its $HOME. What exactly is being stored here and how is it related to the Wikimedia movement?

@ayounsi @BCornwall Please don't if you don't fully understand the effects of an account rename on all the systems that use developer account/LDAP authentication.

@Krinkle @Ladsgroup Hey. wmf.1 includes this change https://gerrit.wikimedia.org/r/c/mediawiki/core/+/829316 which looks related - could you have a look?

I think I fixed this recently.

In T317163#8218071, @dancy wrote:

Right now the +2 review message is "Approved via scap backport". How about something this instead?

Approved by dancy@deploy1002 using scap backport

There's an instance called deployment-shellbox.deployment-prep.eqiad1.wikimedia.cloud - is that not used to run shellbox for beta?

You need to log out and back in after the membership request was approved. This is a known bug in Striker (T144943) and is mentioned in the talk page message that we send out to all Toolforge users when their membership requests are approved.