The moss-be1001 and moss-be1002 are not currently in production so they can be powered off at any time. The thanos host will need more time (roughly a few weeks) as it requires a few steps to adjust replication and take one host out of the cluster.

@Papaul The moss-be2001 and moss-be2002 are not currently in production so they can be powered off at any time. The thanos host will need more time (roughly a few weeks) as it requires a few steps to adjust replication and take one host out of the cluster.

Possibly still wanted / needed, but not really planned at this time.

@Varnent After chatting about this some more, how about we do the following:

@Varnent could we get a clarification of the timeline for this request? The description says end of this month and your recent comment mentions a delay into July.

@Jelto Is there anything else that needs to happen here?

Matthew is out until next Monday, I reached out to Filippo to see if I can provide any details this week on which Thanos host we'll be using.

Correction, Matthew will be back next Monday, I reached out to Filippo to see if I can provide any details this week.

@Papaul Matthew is back on Monday, we'll get back to you then.

Based on the mention of nodetool in the error log it's quite likely to be related to https://phabricator.wikimedia.org/T309736.

@Papaul Is the April 11th date fixed?

Thanks, let us know when the database is created, so we can sanitize it.

@jcrespo Looks good, I made a few formatting changes, see what you think. There's one sentence that may be incomplete: "If there are errors (e.g. download or decryption fails)."

@Jenlenfantwright @LNguyen This task changed state twice (in February and in March) despite a lack of substantial updates since November last year. Is there a record of any discussions or decisions that is available elsewhere?

dumpsdata1007 is running Bullseye BTW for anyone else watching from the sidelines.

@MoritzMuehlenhoff both DB and Backup tooling work is completed so at this point we are ready to go ahead and upgrade the Cumin hosts.

CC @akosiaris and @KOfori for visibility

Is this a blocker for any production work? If yes, what are the time expectations for this to happen?

Thanks, let us know when the database is created, so we can sanitize it.

Sounds good to me, CC'ing @MatthewVernon for visibility.

Fine by me.

Thanks for the explanation. Approved.

Is gitlab-roots actually needed? Maybe the group does something else than the name would suggest but I thought root access on Gitlab hosts was restricted to SRE?

Yesterday there was a question from @Kormat about the Zarcillo DB (on db1115) and what would happen to tools that have its location hardcoded during the reimage. Do we need a "no-maintenance" window?

Removing assignment as I don't believe Manuel will be looking into this in the near future.

This looks like a Mediawiki question so while we (DBA) would likely be interested in the answer, I don't think we are the best team to provide one. I'll tag Platform Engineering and cc @Ladsgroup who may have an opinion.

@Kormat could you take a look?

Should be preceded by https://phabricator.wikimedia.org/T296022.

Adjusting priority and removing the mention of this being a blocker for Tendril deprecation.

I see codfw is configured with 4 different racks so I don't see why we wouldn't do the same thing here.

Adding @MatthewVernon for the Swift hosts.

Logging update proposal (adding section and host to error log message) from @Volans (so that we have it all in one place): https://gerrit.wikimedia.org/r/c/operations/software/wmfbackups/+/736652

Let's wait until Editing rolls out the changes to all wikis before doing this.

Intriguing. I should have taken a screenshot.

Project created: https://phabricator.wikimedia.org/project/view/5611/

A drive-by comment, since T280042 was mentioned - the conversation in that task moved away from MySQL and towards Cassandra (CC'ing @gmodena).

In my case it's Firefox 93 on Ubuntu 20.04.

As discussed in the meeting, cumin2001 will remain in service until DB tooling is packaged for Bullseye (ETA. mid-Q3 FY21/22). At that point cumin1001 will also be re-imagined to Bullseye.