Page MenuHomePhabricator

Arnoldokoth (arnold)
User

Projects (9)

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Aug 10 2021, 2:29 PM (84 w, 6 d)
Availability
Available
LDAP User
AOkoth
MediaWiki User
AOkoth (WMF) [ Global Accounts ]

Recent Activity

Yesterday

Arnoldokoth added a comment to T324506: Investigate incremental backups for GitLab.

@Jelto I tested this on gitlab1004.

Mon, Mar 27, 4:23 PM · GitLab (Infrastructure), serviceops-collab

Tue, Mar 21

Arnoldokoth claimed T309009: Migrate eventgate check_prometheus checks to alertmanager.
Tue, Mar 21, 2:50 PM · Patch-For-Review, Observability-Alerting, Data-Engineering

Mon, Mar 13

Arnoldokoth moved T330920: VRTS Switchover Process from Incoming to Work in Progress on the serviceops-collab board.
Mon, Mar 13, 3:50 PM · serviceops-collab

Wed, Mar 1

Arnoldokoth updated the task description for T330920: VRTS Switchover Process.
Wed, Mar 1, 6:06 PM · serviceops-collab
Arnoldokoth created T330920: VRTS Switchover Process.
Wed, Mar 1, 6:04 PM · serviceops-collab

Feb 14 2023

Arnoldokoth added a comment to T323515: Create VRTS Replica VM in codfw.

@Jelto Thanks for the heads up. I had down timed alerts on this host since it's still a work in progress. I think the downtime expired today but I extended it.

Feb 14 2023, 7:55 PM · vrts, Znuny, serviceops-collab

Jan 31 2023

Arnoldokoth added a comment to T324506: Investigate incremental backups for GitLab.

I have some concerns because the docs state The chosen previous backup is overwritten.. So we might see quite a lot of IO here for extracting the tar archive and writing it again. But tests will show that.

Jan 31 2023, 6:33 PM · GitLab (Infrastructure), serviceops-collab

Jan 23 2023

Arnoldokoth closed T317059: Setup VRTS on WMCS as Resolved.
Jan 23 2023, 4:14 PM · serviceops-collab, vrts

Dec 19 2022

Arnoldokoth closed T324142: Database grants for readonly VRTS user, a subtask of T323515: Create VRTS Replica VM in codfw, as Resolved.
Dec 19 2022, 4:42 PM · vrts, Znuny, serviceops-collab
Arnoldokoth closed T324142: Database grants for readonly VRTS user as Resolved.
Dec 19 2022, 4:42 PM · DBA, vrts, Znuny, serviceops-collab

Dec 15 2022

Arnoldokoth added a comment to T324142: Database grants for readonly VRTS user.

Thanks @Marostegui I can confirm that this works on otrs1001. I trust that it will work on the other host but I have to install the mysql-client. I will test on vrts2001 after making the necessary Puppet changes.

Dec 15 2022, 5:08 PM · DBA, vrts, Znuny, serviceops-collab

Dec 14 2022

Arnoldokoth added a comment to T324142: Database grants for readonly VRTS user.

Actually, just scratch everything I've said so far... We might not need a new database user. Could we just have the current user otrs be able to connnect to m2-slave in both data centers?

aokoth@otrs1001:~$ mysql -h m2-slave.eqiad.wmnet -u otrs -p otrs -P 3323
ERROR 1045 (28000): Access denied for user 'otrs'@'10.64.16.39' (using password: YES)
Dec 14 2022, 8:26 PM · DBA, vrts, Znuny, serviceops-collab
Arnoldokoth added a comment to T324142: Database grants for readonly VRTS user.

Hi @Marostegui We did some testing and the outcome was we found we could connect to the m2-master on both hosts but can't connect to the m2-slave on either server. Despite the fact that the new user has read-only grants, we would prefer that the non-active host connects to the secondary database server as well.

Dec 14 2022, 8:17 PM · DBA, vrts, Znuny, serviceops-collab

Dec 9 2022

Arnoldokoth added a comment to T324142: Database grants for readonly VRTS user.

@Marostegui Yeah, that's okay. I'd like the username vrts.

Dec 9 2022, 5:51 PM · DBA, vrts, Znuny, serviceops-collab

Dec 8 2022

Arnoldokoth closed T324030: Site: codfw VM for VRTS as Resolved.
Dec 8 2022, 8:49 PM · vm-requests, Infrastructure-Foundations, SRE
Arnoldokoth closed T324030: Site: codfw VM for VRTS, a subtask of T323515: Create VRTS Replica VM in codfw, as Resolved.
Dec 8 2022, 8:48 PM · vrts, Znuny, serviceops-collab
Arnoldokoth added a comment to T324030: Site: codfw VM for VRTS.

Resolved: https://netbox.wikimedia.org/virtualization/virtual-machines/539/

Dec 8 2022, 8:48 PM · vm-requests, Infrastructure-Foundations, SRE

Dec 7 2022

Arnoldokoth added a comment to T324142: Database grants for readonly VRTS user.

@Marostegui Yeah, that's exactly what I need but not on the primary database host. We are setting up a new failover VM for VRTS in codfw (vrts2001). The current VRTS instance (otrs1001) connects to the m2-master host in eqiad. So we need a new user that can connect to m2-slave in codfw and said user should only have SELECT grants (I'm not sure if this is implied by the fact that it connects to the secondary host).

Dec 7 2022, 5:39 PM · DBA, vrts, Znuny, serviceops-collab

Dec 5 2022

Arnoldokoth added a comment to T324506: Investigate incremental backups for GitLab.

As an idea, I was combing through the docs (https://docs.gitlab.com/ee/raketasks/backup_gitlab.html#incremental-repository-backups) and noticed Gitlab supports incremental backups:

Dec 5 2022, 8:10 PM · GitLab (Infrastructure), serviceops-collab
Arnoldokoth claimed T324506: Investigate incremental backups for GitLab.
Dec 5 2022, 8:08 PM · GitLab (Infrastructure), serviceops-collab
Arnoldokoth created T324506: Investigate incremental backups for GitLab.
Dec 5 2022, 8:07 PM · GitLab (Infrastructure), serviceops-collab
Arnoldokoth added a comment to T324142: Database grants for readonly VRTS user.

@Marostegui Yeah, that would be okay.

Dec 5 2022, 4:33 PM · DBA, vrts, Znuny, serviceops-collab
Arnoldokoth moved T324142: Database grants for readonly VRTS user from Incoming to Work in Progress on the serviceops-collab board.
Dec 5 2022, 4:25 PM · DBA, vrts, Znuny, serviceops-collab
Arnoldokoth moved T323138: Mail Server For VRTS On WMCS from Work in Progress to Consultation on the serviceops-collab board.
Dec 5 2022, 4:24 PM · vrts, serviceops-collab
Arnoldokoth added a project to T324142: Database grants for readonly VRTS user: DBA.
Dec 5 2022, 4:17 PM · DBA, vrts, Znuny, serviceops-collab

Nov 30 2022

Arnoldokoth created T324142: Database grants for readonly VRTS user.
Nov 30 2022, 7:39 PM · DBA, vrts, Znuny, serviceops-collab
Arnoldokoth changed the status of T324030: Site: codfw VM for VRTS, a subtask of T323515: Create VRTS Replica VM in codfw, from Open to In Progress.
Nov 30 2022, 7:38 PM · vrts, Znuny, serviceops-collab
Arnoldokoth changed the status of T324030: Site: codfw VM for VRTS from Open to In Progress.
Nov 30 2022, 7:37 PM · vm-requests, Infrastructure-Foundations, SRE
Arnoldokoth added a subtask for T323515: Create VRTS Replica VM in codfw: T324030: Site: codfw VM for VRTS.
Nov 30 2022, 7:36 PM · vrts, Znuny, serviceops-collab
Arnoldokoth added a parent task for T324030: Site: codfw VM for VRTS: T323515: Create VRTS Replica VM in codfw.
Nov 30 2022, 7:36 PM · vm-requests, Infrastructure-Foundations, SRE
Arnoldokoth added a comment to T324030: Site: codfw VM for VRTS.
Ready to create Ganeti VM vrts2001.codfw.wmnet in the codfw cluster on group C with 4 vCPUs, 8GB of RAM, 25GB of disk in the private network.
Nov 30 2022, 7:05 PM · vm-requests, Infrastructure-Foundations, SRE

Nov 29 2022

Arnoldokoth created T324030: Site: codfw VM for VRTS.
Nov 29 2022, 3:50 PM · vm-requests, Infrastructure-Foundations, SRE

Nov 21 2022

Arnoldokoth moved T303190: enhance Znuny (otrs) alerting from Backlog to Incoming on the serviceops-collab board.
Nov 21 2022, 4:48 PM · SRE-Sprint-Week-Sustainability-March2023, serviceops-collab, Sustainability (Incident Followup), Znuny
Arnoldokoth added projects to T323515: Create VRTS Replica VM in codfw: Znuny, vrts.
Nov 21 2022, 4:47 PM · vrts, Znuny, serviceops-collab
Arnoldokoth created T323515: Create VRTS Replica VM in codfw.
Nov 21 2022, 4:46 PM · vrts, Znuny, serviceops-collab

Nov 15 2022

Arnoldokoth created T323138: Mail Server For VRTS On WMCS.
Nov 15 2022, 4:58 PM · vrts, serviceops-collab

Oct 24 2022

Arnoldokoth reopened T303272: Move VTRS db passwords to a different hiera location as "Open".
Oct 24 2022, 7:15 PM · Patch-For-Review, serviceops-collab, Znuny, SRE
Arnoldokoth added a comment to T303272: Move VTRS db passwords to a different hiera location.

Hey @jbond Seems like the merge broke Puppet on otrs1001.eqiad.wmnet. It fails with the following error:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Function Call, Failed to parse template vrts/exim4.conf.vrts.erb:
  Filepath: /etc/puppet/modules/vrts/templates/exim4.conf.vrts.erb
  Line: 22
  Detail: undefined method `unwrap' for "<redacted>":String
 (file: /etc/puppet/modules/vrts/manifests/mail.pp, line: 41, column: 20) on node otrs1001.eqiad.wmnet
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
Oct 24 2022, 7:15 PM · Patch-For-Review, serviceops-collab, Znuny, SRE

Oct 7 2022

Arnoldokoth added a comment to T319014: Requesting access to analytics-privatedata-users for Lucas Werkmeister.

Hey @Lucas_Werkmeister_WMDE Yeah, I was actually debating whether to remove that checkbox but I'll just leave it unchecked since it's not needed in this case. It's resolved now. Feel free to close the ticket.

Oct 7 2022, 7:00 PM · SRE, SRE-Access-Requests
Arnoldokoth triaged T319014: Requesting access to analytics-privatedata-users for Lucas Werkmeister as Medium priority.
Oct 7 2022, 2:42 PM · SRE, SRE-Access-Requests

Oct 6 2022

Arnoldokoth triaged T319326: Requesting access to analytics-privatedata-users for hshaikh and ptiwary as Medium priority.
Oct 6 2022, 6:31 PM · SRE, SRE-Access-Requests
Arnoldokoth assigned T319014: Requesting access to analytics-privatedata-users for Lucas Werkmeister to karapayneWMDE.
Oct 6 2022, 6:14 PM · SRE, SRE-Access-Requests
Arnoldokoth changed the status of T319326: Requesting access to analytics-privatedata-users for hshaikh and ptiwary from Open to In Progress.
Oct 6 2022, 6:10 PM · SRE, SRE-Access-Requests
Arnoldokoth updated the task description for T319326: Requesting access to analytics-privatedata-users for hshaikh and ptiwary.
Oct 6 2022, 6:09 PM · SRE, SRE-Access-Requests
Arnoldokoth moved T319014: Requesting access to analytics-privatedata-users for Lucas Werkmeister from Untriaged to Manager/NDA Approval/Confirmation on the SRE-Access-Requests board.
Oct 6 2022, 6:07 PM · SRE, SRE-Access-Requests
Arnoldokoth moved T319056: Requesting access to analytics-privatedata-users for Wenjun Fan from Untriaged to Awaiting User Input on the SRE-Access-Requests board.
Oct 6 2022, 6:06 PM · SRE, SRE-Access-Requests
Arnoldokoth updated subscribers of T319056: Requesting access to analytics-privatedata-users for Wenjun Fan.
Oct 6 2022, 5:55 PM · SRE, SRE-Access-Requests
Arnoldokoth updated subscribers of T319056: Requesting access to analytics-privatedata-users for Wenjun Fan.

Hey @AnnWF Kindly sign this https://phabricator.wikimedia.org/L3

Oct 6 2022, 5:54 PM · SRE, SRE-Access-Requests
Arnoldokoth added a comment to T318626: Grant Access to wmf for "Stef Dunlap".

I have added to the wmf-nda group as well. Thanks @Aklapper

Oct 6 2022, 4:10 PM · SRE, LDAP-Access-Requests
Arnoldokoth added a comment to T318729: Grant Access to wmf for MHorsey.

Ooh, sorry I missed that step. I have added you to the wmf-nda group as well. Thanks @Aklapper

Oct 6 2022, 4:09 PM · SRE, LDAP-Access-Requests
Arnoldokoth added a member for WMF-NDA: SDunlap.
Oct 6 2022, 4:08 PM
Arnoldokoth added a member for WMF-NDA: MHorsey-WMF.
Oct 6 2022, 4:06 PM

Oct 5 2022

Arnoldokoth updated the task description for T319014: Requesting access to analytics-privatedata-users for Lucas Werkmeister.
Oct 5 2022, 9:19 PM · SRE, SRE-Access-Requests
Arnoldokoth updated subscribers of T319014: Requesting access to analytics-privatedata-users for Lucas Werkmeister.

Hey @Ottomata @karapayneWMDE Kindly approve.

Oct 5 2022, 9:18 PM · SRE, SRE-Access-Requests
Arnoldokoth moved T318807: Requesting access to analytics-private-users for Slavina Stefanova from Manager/NDA Approval/Confirmation to Ready To Go on the SRE-Access-Requests board.
Oct 5 2022, 9:15 PM · SRE, SRE-Access-Requests
Arnoldokoth moved T318873: Requesting access to analytics-privatedata-users for Greg Grossmeier from Manager/NDA Approval/Confirmation to Ready To Go on the SRE-Access-Requests board.
Oct 5 2022, 9:15 PM · SRE, SRE-Access-Requests
Arnoldokoth added a comment to T318873: Requesting access to analytics-privatedata-users for Greg Grossmeier.

@greg This is resolved. Feel free to close the ticket if everything is good on your side.

Oct 5 2022, 9:15 PM · SRE, SRE-Access-Requests
Arnoldokoth added a comment to T318807: Requesting access to analytics-private-users for Slavina Stefanova.

@Slst2020 This is resolved. Feel free to close the ticket if everything is good on your end.

Oct 5 2022, 8:48 PM · SRE, SRE-Access-Requests
Arnoldokoth updated the task description for T318873: Requesting access to analytics-privatedata-users for Greg Grossmeier.
Oct 5 2022, 8:41 PM · SRE, SRE-Access-Requests
Arnoldokoth updated the task description for T318873: Requesting access to analytics-privatedata-users for Greg Grossmeier.
Oct 5 2022, 8:40 PM · SRE, SRE-Access-Requests
Arnoldokoth updated the task description for T318807: Requesting access to analytics-private-users for Slavina Stefanova.
Oct 5 2022, 8:34 PM · SRE, SRE-Access-Requests
Arnoldokoth added a comment to T318626: Grant Access to wmf for "Stef Dunlap".
aokoth@mwmaint1002:~$ ldapsearch -x cn=wmf | grep "kindrobot"
member: uid=kindrobot,ou=people,dc=wikimedia,dc=org
Oct 5 2022, 8:30 PM · SRE, LDAP-Access-Requests
Arnoldokoth changed the status of T318626: Grant Access to wmf for "Stef Dunlap" from Open to In Progress.
Oct 5 2022, 7:56 PM · SRE, LDAP-Access-Requests
Arnoldokoth added a comment to T318729: Grant Access to wmf for MHorsey.
aokoth@mwmaint1002:~$ ldapsearch -x cn=wmf | grep "mhorsey"
member: uid=mhorsey,ou=people,dc=wikimedia,dc=org
Oct 5 2022, 7:47 PM · SRE, LDAP-Access-Requests
Arnoldokoth changed the status of T318729: Grant Access to wmf for MHorsey from Open to In Progress.
Oct 5 2022, 7:30 PM · SRE, LDAP-Access-Requests
Arnoldokoth updated the task description for T318807: Requesting access to analytics-private-users for Slavina Stefanova.
Oct 5 2022, 2:02 PM · SRE, SRE-Access-Requests
Arnoldokoth closed T319274: Remove old production ssh key for RelEng user as Resolved.
Oct 5 2022, 1:49 PM · SRE-Access-Requests, SRE
Arnoldokoth moved T318807: Requesting access to analytics-private-users for Slavina Stefanova from Untriaged to Manager/NDA Approval/Confirmation on the SRE-Access-Requests board.
Oct 5 2022, 1:48 PM · SRE, SRE-Access-Requests
Arnoldokoth moved T318873: Requesting access to analytics-privatedata-users for Greg Grossmeier from Untriaged to Manager/NDA Approval/Confirmation on the SRE-Access-Requests board.
Oct 5 2022, 1:48 PM · SRE, SRE-Access-Requests
Arnoldokoth updated the task description for T318873: Requesting access to analytics-privatedata-users for Greg Grossmeier.
Oct 5 2022, 1:47 PM · SRE, SRE-Access-Requests
Arnoldokoth updated subscribers of T318873: Requesting access to analytics-privatedata-users for Greg Grossmeier.

Hey @greg Yeah, Lisa G is your manager (confirmed on Namely). So will need approval from her (@Lgruwell-WMF ) as well as @Ottomata or @odimitrijevic

Oct 5 2022, 1:47 PM · SRE, SRE-Access-Requests

Oct 4 2022

Arnoldokoth updated the task description for T318807: Requesting access to analytics-private-users for Slavina Stefanova.
Oct 4 2022, 8:41 PM · SRE, SRE-Access-Requests
Arnoldokoth updated the task description for T318807: Requesting access to analytics-private-users for Slavina Stefanova.
Oct 4 2022, 8:19 PM · SRE, SRE-Access-Requests
Arnoldokoth updated the task description for T318807: Requesting access to analytics-private-users for Slavina Stefanova.
Oct 4 2022, 8:11 PM · SRE, SRE-Access-Requests
Arnoldokoth updated the task description for T318807: Requesting access to analytics-private-users for Slavina Stefanova.
Oct 4 2022, 8:10 PM · SRE, SRE-Access-Requests
Arnoldokoth updated subscribers of T318807: Requesting access to analytics-private-users for Slavina Stefanova.

Hey, @Ottomata You are listed as one of the approvers for members joining this list. Kindly approve.

Oct 4 2022, 5:07 PM · SRE, SRE-Access-Requests
Arnoldokoth moved T318104: Requesting access to Analytics for devnull from Untriaged to Awaiting User Input on the SRE-Access-Requests board.
Oct 4 2022, 3:27 PM · SRE, SRE-Access-Requests
Arnoldokoth triaged T319274: Remove old production ssh key for RelEng user as Medium priority.
Oct 4 2022, 3:17 PM · SRE-Access-Requests, SRE
Arnoldokoth changed the status of T319274: Remove old production ssh key for RelEng user from Open to In Progress.
Oct 4 2022, 3:16 PM · SRE-Access-Requests, SRE

Sep 20 2022

Arnoldokoth triaged T317059: Setup VRTS on WMCS as Medium priority.
Sep 20 2022, 2:30 PM · serviceops-collab, vrts
Arnoldokoth moved T317059: Setup VRTS on WMCS from Incoming 🐫 to Doing 😎 on the serviceops board.
Sep 20 2022, 2:30 PM · serviceops-collab, vrts

Sep 5 2022

Arnoldokoth updated the task description for T317059: Setup VRTS on WMCS.
Sep 5 2022, 6:18 PM · serviceops-collab, vrts
Arnoldokoth created T317059: Setup VRTS on WMCS.
Sep 5 2022, 6:16 PM · serviceops-collab, vrts

Sep 4 2022

Arnoldokoth closed T296713: Document and test failover for GitLab and GitLab Replica as Resolved.
Sep 4 2022, 9:10 AM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, serviceops

Sep 1 2022

Arnoldokoth added a comment to T316903: vrts - spamassassin icinga alerts.
Sep 01 20:19:06 otrs1001 systemd[1]: spamassassin_updates.service: Succeeded
Sep 1 2022, 8:21 PM · serviceops, serviceops-collab, vrts
Arnoldokoth added a comment to T316903: vrts - spamassassin icinga alerts.

Hey @Dzahn Wasn't this patch reverted?

Sep 1 2022, 8:02 PM · serviceops, serviceops-collab, vrts

Aug 25 2022

Arnoldokoth added a comment to T316148: Request increased quota for devtools Cloud VPS project.

Hi @rook Sorry. I hadn't noticed that the number of instances should also be increased as part of the quota. Given the resources that were added, I think we need to increase that by two since it fits the two smallest instance flavors.

Aug 25 2022, 4:57 PM · cloud-services-team (Kanban), Cloud-VPS (Quota-requests)
Arnoldokoth reopened T316148: Request increased quota for devtools Cloud VPS project as "Open".
Aug 25 2022, 4:55 PM · cloud-services-team (Kanban), Cloud-VPS (Quota-requests)
Dzahn awarded T316148: Request increased quota for devtools Cloud VPS project a Like token.
Aug 25 2022, 3:10 PM · cloud-services-team (Kanban), Cloud-VPS (Quota-requests)

Aug 24 2022

Arnoldokoth added a comment to T316148: Request increased quota for devtools Cloud VPS project.

@taavi Scratch that. I just confirmed we don't need the floating IP. Thanks.

Aug 24 2022, 9:20 PM · cloud-services-team (Kanban), Cloud-VPS (Quota-requests)
Arnoldokoth updated the task description for T316148: Request increased quota for devtools Cloud VPS project.
Aug 24 2022, 6:52 PM · cloud-services-team (Kanban), Cloud-VPS (Quota-requests)
Arnoldokoth updated the task description for T316148: Request increased quota for devtools Cloud VPS project.
Aug 24 2022, 6:52 PM · cloud-services-team (Kanban), Cloud-VPS (Quota-requests)
Arnoldokoth created T316148: Request increased quota for devtools Cloud VPS project.
Aug 24 2022, 6:52 PM · cloud-services-team (Kanban), Cloud-VPS (Quota-requests)

Aug 19 2022

Arnoldokoth added a comment to T303272: Move VTRS db passwords to a different hiera location.

Hey @Kormat @jbond I was wondering if this was resolved. I noticed the file now references some vrts passwords.

# less modules/profile/templates/mariadb/grants/production-m2.sql.erb  | grep "vrts"
IDENTIFIED BY '<%= @vrts_exim_database_pass %>'
IDENTIFIED BY '<%= @vrts_database_pw %>'
IDENTIFIED BY '<%= @vrts_exim_database_pass %>'
IDENTIFIED BY '<%= @vrts_database_pw %>'
IDENTIFIED BY '<%= @vrts_exim_database_pass %>'
IDENTIFIED BY '<%= @vrts_database_pw %>'
Aug 19 2022, 6:37 PM · Patch-For-Review, serviceops-collab, Znuny, SRE

Aug 8 2022

Arnoldokoth added a comment to T296713: Document and test failover for GitLab and GitLab Replica.

https://gerrit.wikimedia.org/r/c/operations/puppet/+/820563 @LSobanski This was the fix for the SSL issue. The record gitlab-replica-old had been used before for this purpose but it was removed when testing was complete.

Aug 8 2022, 8:27 PM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, serviceops
Jelto awarded T296713: Document and test failover for GitLab and GitLab Replica a Like token.
Aug 8 2022, 3:33 PM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, serviceops

Aug 4 2022

Arnoldokoth updated subscribers of T296713: Document and test failover for GitLab and GitLab Replica.

Thanks to @Dzahn the SSL issue is now resolved.

Aug 4 2022, 9:42 PM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, serviceops
Arnoldokoth added a comment to T296713: Document and test failover for GitLab and GitLab Replica.

@Jelto I managed to switch over the hosts i.e. gitlab1003 and gitlab2002. The current replica is now gitlab2002. Though the old replica is currently experiencing some SSL issues and I'm not entirely sure why. Perhaps I missed a configuration step but didn't do much on the old host other than disabling and re-enabling puppet.

Aug 4 2022, 8:29 PM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, serviceops
Arnoldokoth added a comment to T296713: Document and test failover for GitLab and GitLab Replica.
# host gitlab-replica-old.wikimedia.org
gitlab-replica-old.wikimedia.org has address 208.80.154.15
gitlab-replica-old.wikimedia.org has IPv6 address 2620:0:861:1:208:80:154:15
# host gitlab-replica.wikimedia.org
gitlab-replica.wikimedia.org has address 208.80.153.8
gitlab-replica.wikimedia.org has IPv6 address 2620:0:860:1:208:80:153:8
Aug 4 2022, 8:26 PM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, serviceops

Aug 1 2022

Arnoldokoth edited projects for T296713: Document and test failover for GitLab and GitLab Replica, added: GitLab (Infrastructure); removed GitLab.
Aug 1 2022, 4:00 PM · Patch-For-Review, GitLab (Infrastructure), serviceops-collab, serviceops