Page MenuHomePhabricator
Create Task
Maniphest T293012

Productionise mc20[38-55]
Closed, ResolvedPublic
Actions

Description

After mc20[38-55] have been racked, we need to put them in production.

  • host mc2037 was created after mc2028 died
  • we should consider if it makes sense to make an exception and renumber these hosts from 2037 so they are in par with eqiad.
  • more details to look out for can be found in T293216

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
WIP: Add mc2038-mc2055operations/puppetproduction+1 -6
mediawiki::mcrouter_wancache: Add mc2038 to memcached clusteroperations/puppetproduction+2 -2
mediawiki::mcrouter_wancache: Add mc2040 & mc2041 to memcached clusteroperations/puppetproduction+4 -4
mediawiki::mcrouter_wancache: Add "mc2041 mc2042" to memcached clusteroperations/puppetproduction+4 -4
mediawiki::mcrouter_wancache: Add mc2044 & mc2043 to memcached clusteroperations/puppetproduction+4 -4
mediawiki::mcrouter_wancache: Add mc2045 & mc2046 to memcached clusteroperations/puppetproduction+4 -4
mediawiki::mcrouter_wancache: Add mc2047 & mc2048 to memcached clusteroperations/puppetproduction+4 -4
mediawiki::mcrouter_wancache: Add mc2049 to memcached clusteroperations/puppetproduction+4 -4
mediawiki::mcrouter_wancache: Add mc2050 to memcached clusteroperations/puppetproduction+2 -2
mediawiki::mcrouter_wancache: Add mc2051 to memcached clusteroperations/puppetproduction+2 -2
mediawiki::mcrouter_wancache: Add mc2052 to memcached clusteroperations/puppetproduction+2 -2
mediawiki::mcrouter_wancache: Add mc2053 to memcached clusteroperations/puppetproduction+2 -2
mediawiki::mcrouter_wancache: Add mc2054 to memcached clusteroperations/puppetproduction+2 -2
mediawiki::mcrouter_wancache: Add mc2055 to memcached clusteroperations/puppetproduction+2 -2
site: Productionise mc20[39-55]operations/puppetproduction+3 -3
redis: Replace mc2024 with mc2038operations/puppetproduction+2 -2
hiera: replace mc2024 with mc2038operations/puppetproduction+1 -1
mcrouter_wancache: Replace mc2024 with mc2038operations/puppetproduction+2 -2
site: Add mc2038 as mediawiki::memcachedoperations/puppetproduction+2 -2
Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedJhancock.wmT313733 Decommission mc20[19-27] and mc20[29-37]
 ResolvedjijikiT293012 Productionise mc20[38-55]
Unknown Object (Task)
 ResolvedPapaulT294962 Q2:(Need By: TBD) rack/setup/install mc20[38-55]
 DuplicateNoneT302218 setup/install mc20[38-55]

Event Timeline

jijiki created this task.Oct 11 2021, 4:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 11 2021, 4:08 PM
jijiki added a subtask: Unknown Object (Task).Oct 11 2021, 4:08 PM
jijiki updated the task description. (Show Details)Oct 14 2021, 4:21 PM
Dzahn changed the status of subtask Unknown Object (Task) from Open to In Progress.Oct 26 2021, 9:13 PM
Dzahn mentioned this in Unknown Object (Task).Oct 26 2021, 9:54 PM
RobH changed the status of subtask Unknown Object (Task) from In Progress to Open.Nov 3 2021, 6:22 PM
Papaul closed subtask Unknown Object (Task) as Resolved.Dec 3 2021, 4:04 PM
akosiaris mentioned this in T302218: setup/install mc20[38-55].Feb 22 2022, 2:18 PM
akosiaris merged a task: T302218: setup/install mc20[38-55].
akosiaris added subscribers: akosiaris, Papaul, Dzahn and 3 others.
akosiaris added a comment.May 13 2022, 10:45 AM

we should consider if it makes sense to make an exception and renumber these hosts from 2037 so they are in par with eqiad.

Niah, that would create confusion. Also, numbers don't need to match up between the 2 DCs, things happen (as in this case).

gerritbot added a comment.May 13 2022, 10:56 AM

Change 791583 had a related patch set uploaded (by Alexandros Kosiaris; author: Alexandros Kosiaris):

[operations/puppet@production] WIP: Add mc2038-mc2055

https://gerrit.wikimedia.org/r/791583

gerritbot added a project: Patch-For-Review.May 13 2022, 10:56 AM
akosiaris mentioned this in T313733: Decommission mc20[19-27] and mc20[29-37].Jul 25 2022, 2:31 PM
RLazarus subscribed.Aug 2 2022, 6:05 PM

Due to T309956 I'm moving ahead with mc2038 early, and using it to replace mc2024 which is currently out of service.

gerritbot added a comment.Aug 2 2022, 6:08 PM

Change 819655 had a related patch set uploaded (by RLazarus; author: RLazarus):

[operations/puppet@production] site: Add mc2038 as mediawiki::memcached

https://gerrit.wikimedia.org/r/819655

gerritbot added a comment.Aug 2 2022, 6:12 PM

Change 819656 had a related patch set uploaded (by RLazarus; author: RLazarus):

[operations/puppet@production] mcrouter_wancache: Replace mc2024 with mc2038

https://gerrit.wikimedia.org/r/819656

ops-monitoring-bot added a comment.Aug 2 2022, 6:18 PM

Icinga downtime and Alertmanager silence (ID=9144f946-b4bf-404f-ac18-b48c723e759c) set by rzl@cumin2002 for 2:00:00 on 1 host(s) and their services with reason: install

mc2038.codfw.wmnet
gerritbot added a comment.Aug 2 2022, 6:20 PM

Change 819655 merged by RLazarus:

[operations/puppet@production] site: Add mc2038 as mediawiki::memcached

https://gerrit.wikimedia.org/r/819655

RLazarus added a comment.Aug 2 2022, 6:42 PM

!log rzl@cumin2002 START - Cookbook sre.hosts.remove-downtime for mc2038.codfw.wmnet
!log rzl@cumin2002 END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mc2038.codfw.wmnet

ops-monitoring-bot added a comment.Aug 2 2022, 6:46 PM

Icinga downtime and Alertmanager silence (ID=819c43c7-89c1-4902-b636-e59fabe9011d) set by rzl@cumin2002 for 1:00:00 on 1 host(s) and their services with reason: install

mc2038.codfw.wmnet
gerritbot added a comment.Aug 2 2022, 7:08 PM

Change 819697 had a related patch set uploaded (by RLazarus; author: RLazarus):

[operations/puppet@production] redis: Replace mc2024 with mc2038

https://gerrit.wikimedia.org/r/819697

ops-monitoring-bot added a comment.Aug 2 2022, 7:17 PM

Icinga downtime and Alertmanager silence (ID=1da0e7c2-1678-4926-858d-8864d2200918) set by rzl@cumin2002 for 1 day, 0:00:00 on 1 host(s) and their services with reason: install

mc2038.codfw.wmnet
gerritbot added a comment.Aug 2 2022, 7:32 PM

Change 819706 had a related patch set uploaded (by Ssingh; author: Ssingh):

[operations/puppet@production] hiera: replace mc2024 with mc2038

https://gerrit.wikimedia.org/r/819706

gerritbot added a comment.Aug 3 2022, 7:33 AM

Change 819656 merged by Giuseppe Lavagetto:

[operations/puppet@production] mcrouter_wancache: Replace mc2024 with mc2038

https://gerrit.wikimedia.org/r/819656

gerritbot added a comment.Aug 3 2022, 12:51 PM

Change 819706 abandoned by Ssingh:

[operations/puppet@production] hiera: replace mc2024 with mc2038

Reason:

already merged

https://gerrit.wikimedia.org/r/819706

jijiki claimed this task.Aug 29 2022, 11:12 AM
jijiki moved this task from Incoming 🐫 to 🙈🙉🙊Backlog on the serviceops board.Sep 28 2022, 2:20 PM
jijiki moved this task from 🙈🙉🙊Backlog to this.quarter 🍕 on the serviceops board.Nov 8 2022, 3:54 PM
jijiki moved this task from this.quarter 🍕 to Doing 😎 on the serviceops board.Dec 5 2022, 6:17 PM
jijiki triaged this task as High priority.Dec 6 2022, 5:42 PM
gerritbot added a comment.Dec 7 2022, 8:36 PM

Change 865736 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] site.pp Productionise mc20[39-55]

https://gerrit.wikimedia.org/r/865736

gerritbot added a comment.Dec 8 2022, 11:12 AM

Change 865736 merged by Effie Mouzeli:

[operations/puppet@production] site: Productionise mc20[39-55]

https://gerrit.wikimedia.org/r/865736

gerritbot added a comment.Dec 8 2022, 11:51 AM

Change 866334 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache Add mc2055 to memcached cluster

https://gerrit.wikimedia.org/r/866334

gerritbot added a comment.Dec 8 2022, 1:51 PM

Change 866334 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2055 to memcached cluster

https://gerrit.wikimedia.org/r/866334

gerritbot added a comment.Dec 8 2022, 3:02 PM

Change 866387 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2054 to memcached cluster

https://gerrit.wikimedia.org/r/866387

gerritbot added a comment.Dec 8 2022, 3:03 PM

Change 866387 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2054 to memcached cluster

https://gerrit.wikimedia.org/r/866387

gerritbot added a comment.Dec 9 2022, 1:05 PM

Change 866587 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2053 to memcached cluster

https://gerrit.wikimedia.org/r/866587

gerritbot added a comment.Dec 9 2022, 1:11 PM

Change 866587 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2053 to memcached cluster

https://gerrit.wikimedia.org/r/866587

gerritbot added a comment.Dec 9 2022, 2:20 PM

Change 866595 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2053 to memcached cluster

https://gerrit.wikimedia.org/r/866595

gerritbot added a comment.Dec 9 2022, 2:22 PM

Change 866595 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2052 to memcached cluster

https://gerrit.wikimedia.org/r/866595

gerritbot added a comment.Dec 12 2022, 9:42 AM

Change 867119 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2051 to memcached cluster

https://gerrit.wikimedia.org/r/867119

gerritbot added a comment.Dec 12 2022, 9:52 AM

Change 867119 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2051 to memcached cluster

https://gerrit.wikimedia.org/r/867119

gerritbot added a comment.Dec 12 2022, 11:42 AM

Change 867140 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2050 to memcached cluster

https://gerrit.wikimedia.org/r/867140

gerritbot added a comment.Dec 12 2022, 1:22 PM

Change 867140 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2050 to memcached cluster

https://gerrit.wikimedia.org/r/867140

gerritbot added a comment.Dec 12 2022, 2:12 PM

Change 867173 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2049 to memcached cluster

https://gerrit.wikimedia.org/r/867173

gerritbot added a comment.Dec 12 2022, 2:44 PM

Change 867181 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2047 & mc2048 to memcached cluster

https://gerrit.wikimedia.org/r/867181

gerritbot added a comment.Dec 12 2022, 2:50 PM

Change 867173 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2049 to memcached cluster

https://gerrit.wikimedia.org/r/867173

Papaul unsubscribed.Dec 12 2022, 3:51 PM
Dzahn unsubscribed.Dec 12 2022, 3:52 PM
gerritbot added a comment.Dec 12 2022, 4:49 PM

Change 867181 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2047 & mc2048 to memcached cluster

https://gerrit.wikimedia.org/r/867181

gerritbot added a comment.Dec 13 2022, 11:21 AM

Change 867545 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2045 & mc2046 to memcached cluster

https://gerrit.wikimedia.org/r/867545

gerritbot added a comment.Dec 13 2022, 11:35 AM

Change 867545 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2045 & mc2046 to memcached cluster

https://gerrit.wikimedia.org/r/867545

gerritbot added a comment.Dec 13 2022, 1:24 PM

Change 867588 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2044 & mc2043 to memcached cluster

https://gerrit.wikimedia.org/r/867588

gerritbot added a comment.Dec 13 2022, 1:26 PM

Change 867588 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2044 & mc2043 to memcached cluster

https://gerrit.wikimedia.org/r/867588

gerritbot added a comment.Dec 13 2022, 2:29 PM

Change 867598 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add "mc2041 mc2042" to memcached cluster

https://gerrit.wikimedia.org/r/867598

gerritbot added a comment.Dec 13 2022, 2:37 PM

Change 867598 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add "mc2041 mc2042" to memcached cluster

https://gerrit.wikimedia.org/r/867598

gerritbot added a comment.Dec 13 2022, 3:27 PM

Change 867649 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2040 & mc2041 to memcached cluster

https://gerrit.wikimedia.org/r/867649

gerritbot added a comment.Dec 13 2022, 3:34 PM

Change 867649 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2040 & mc2041 to memcached cluster

https://gerrit.wikimedia.org/r/867649

gerritbot added a comment.Dec 13 2022, 4:12 PM

Change 867653 had a related patch set uploaded (by Effie Mouzeli; author: Effie Mouzeli):

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2038 to memcached cluster

https://gerrit.wikimedia.org/r/867653

gerritbot added a comment.Dec 13 2022, 4:15 PM

Change 867653 merged by Effie Mouzeli:

[operations/puppet@production] mediawiki::mcrouter_wancache: Add mc2038 to memcached cluster

https://gerrit.wikimedia.org/r/867653

jijiki closed this task as Resolved.Dec 14 2022, 5:31 PM

All hosts are in production.

akosiaris added a parent task: T313733: Decommission mc20[19-27] and mc20[29-37].Dec 15 2022, 12:19 PM
gerritbot added a comment.Oct 21 2024, 11:45 AM

Change #791583 abandoned by Effie Mouzeli:

[operations/puppet@production] WIP: Add mc2038-mc2055

Reason:

hosts have been added

https://gerrit.wikimedia.org/r/791583

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits