I just moved integration-slave-docker-1038 and integration-slave-docker-1034 to eqiad1-r and repooled them. They look OK to me, so far.

I've moved several docker-XXXX jessie-XXXX nodes and that seems to work fine. So I'll finish up with those on my own.

The next VMs of interest to me are integration-cumin and integration-publishing which might be more disruptive.

Change 474279 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Explain why integration-publishing IP is hardcoded

https://gerrit.wikimedia.org/r/474279

So for integration-publishing, its used from contint1001 which is a production machine. The server thus can not resolve the DNS entry integration-publishing.integration.eqiad.wmflabs. and we have hardcoded the instance IP address in the Jenkins jobs.

I think the best way to migrate is to create a fresh new instance in the new region, once we have the IP adress migrate the Jenkins job to use that new instance then we can discard the old one.

For castor02.integration.eqiad.wmflabs, I will have to double check, but I think they always use the DNS entry and it is always run from labs instance. So should be fine to migrate. But, castor02 is usually a SPOF for most of the CI jobs. We would probably want to create a new one, rsync the caches there and switch the jobs.

For the integration-slave hosts, some have been migrated this week with @thcipriani , they seem all fine. Or at least, I haven't been reported any issue related to the migration. I am fine having the other integration-slave ones to be migrated. Kudos!

Change 474279 merged by Andrew Bogott:
[integration/config@master] Explain why integration-publishing IP is hardcoded

https://gerrit.wikimedia.org/r/474279

Mentioned in SAL (#wikimedia-releng) [2018-11-19T13:21:10Z] <hashar> Created integration-publishing02 172.16.4.5 for WMCS region migration # T208803

Change 474689 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Switch to integration-publishing02

https://gerrit.wikimedia.org/r/474689

Mentioned in SAL (#wikimedia-releng) [2018-11-19T13:47:40Z] <hashar> Shutdown integration-publishing , replaced by integration-publishing02 # T208803

Change 474689 merged by jenkins-bot:
[integration/config@master] Switch to integration-publishing02

https://gerrit.wikimedia.org/r/474689

I have switched the CI jobs from integration-publishing to integration-publishing02 in the new region. I have shutdown the instance, will delete it in a few days.

For castor02, we always use the DNS entry, the only exception being in the Jenkins configuration for the node. So we can migrate like other instances. That will cause a slight downtime of CI while it is being done.

Mentioned in SAL (#wikimedia-releng) [2018-11-20T14:59:24Z] <hashar> created integration-castor03.integration.eqiad.wmflabs intended as a replacement for castor02 | T208803

I will prepare the migration to integration-castor03 later tonight and attempt to do the migration tomorrow or on Thursday (easier for me)

Eventually I felt migrating castor03 just before ThanksGiving to be a terrible idea. Postponed to next week.

Change 476213 had a related patch set uploaded (by Hashar; owner: Hashar):
[integration/config@master] Migrate Castor to integration-castor03

https://gerrit.wikimedia.org/r/476213

Change 476213 merged by jenkins-bot:
[integration/config@master] Migrate Castor to integration-castor03

https://gerrit.wikimedia.org/r/476213

Mentioned in SAL (#wikimedia-releng) [2018-11-28T08:45:37Z] <hashar> Switching Jenkins job cache to integration-castor03 with AN EMPTY CACHE | T208803

CI jobs have been migrated to use integration-castor03. I am keeping castor02 around just in case.

Last rsync entry for castor02:

Nov 28 08:38:17 castor02 rsyncd[19693]: rsync on caches/pywikibot-core/master/pywikibot-core-tox-nose34-docker/ from integration-slave-docker-1037.integration.eqiad.wmflabs (10.68.21.155)

integration-castor03 cache is filling up properly.

doc.wikimedia.org has received updates hence the new publishing instance is working.

Deleting castor02 and integration-publishing

Mentioned in SAL (#wikimedia-releng) [2018-11-29T08:35:37Z] <hashar> Deleted instances castor02 and integration-publishing , replaced by new instances in the new WMCS region | T208803

Thanks!

I would like to move integration-puppetmaster01 and webperformance sometime soon. Any objection?

webperformance should be easy to migrate, it runs some jobs for the performance team but apart from that it is a regular Jenkins slave. Just have to remember to update its IP address in the Jenkins slave configuration.

integration-puppetmaster01 that potentially can end up being messy. But I think the instance have puppet.conf pointing to the hostname. Given DNS is adjusted, it should just work. I am not too worried :-)

Note: releng is having an offsite this week so we have little ability unfortunately.

I've now moved everything in this project except for jenkinstest.integration.eqiad.wmflabs. I can't tell what that is -- should I move it blindly, or delete it, or does it need special care?

In T208803#4806941, @Andrew wrote:

I've now moved everything in this project except for jenkinstest.integration.eqiad.wmflabs. I can't tell what that is -- should I move it blindly, or delete it, or does it need special care?

I have a vague memory of this and I don't think there is any special care needed. I am reasonably certain that nothing depends on this instance and it can be moved blindly at will.

Thank you to have completed the migration!

For jenkintest that is most probably a one off dev instance I have created. There are evidence I have used it for T183569: npm 1.4.21 can't use a http proxy. There are no jobs running on it so it can be deleted.

Success! \o/

Mentioned in SAL (#wikimedia-releng) [2018-12-11T10:03:44Z] <hashar> Deleting unused instance jenkinstest.integration.eqiad.wmflabs | T208803