⚓ T256396 Create a runbook for switching CI master

		Status	Subtype	Assigned	Task
		Open		None	T256422 switch contint prod server back from contint2001 to contint1001
		Open		• hashar	T256396 Create a runbook for switching CI master

• hashar created this task.Jun 25 2020, 5:01 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 25 2020, 5:01 PM

Jdforrester-WMF moved this task from Untriaged to Next on the Continuous-Integration-Infrastructure board.Jun 25 2020, 5:03 PM

Dzahn mentioned this in T256422: switch contint prod server back from contint2001 to contint1001.Jun 25 2020, 8:18 PM

Literally extract the information from T224591

The runbook address the issues we had with uid/gid changing during the transfer which then require painful chmod all over the place. The reason is the quickdata copy puppet classes that spawns rsync within a chroot which in turns prevent uid/gid mapping.

• hashar added a parent task: T256422: switch contint prod server back from contint2001 to contint1001.Sep 30 2020, 9:11 PM

• hashar moved this task from INBOX to New Work on the Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)) board.Oct 14 2020, 8:34 AM

• hashar triaged this task as High priority.Nov 4 2020, 6:23 PM

• hashar updated the task description. (Show Details)Dec 10 2020, 9:52 AM

Tyler bootstrapped https://www.mediawiki.org/wiki/Continuous_integration/Data_center_switch and I have edited it with the basic skeleton of how to do a switch over. The biggest devils are:

how to transfer the massive amount of Jenkins build history. Suggestions:
- tar over netcat
- https://wikitech.wikimedia.org/wiki/Transfer.py
- https://wikitech.wikimedia.org/wiki/MariaDB/ImportTableSpace
transfer the Jenkins and Zuul states keeping the proper user/group ownership (rsync quickdata copy shallow them since rsync runs under a chroot and can not do the mapping)
describe the puppet settings and what to change. Ultimately we would want it to be just a Puppet role change instead of messing with hiera/puppet.