Page MenuHomePhabricator

Provide cross-dc redundancy (active-active or active-passive) to all important misc services
Open, MediumPublic

Description

There are some non-core, non-mediawiki services that may or may not be desired, or may or may not be ready to switchover to codfw. These are not part of the goal, but it would be nice to know, for each one:

  1. This service is ready to switchover
  2. This service is not ready, but it would be desired
  3. This service is not either ready or intended to be switchover
ServiceStateComment
Cloud-related servicesNot ready or intended to switchover
AnalyticsNot ready or intended to switchover
DumpsNot ready or intended to switchover
PlanetReady https://gerrit.wikimedia.org/r/347892
fluorine / mwlog1001Ready T123728: replace fluorine with mwlog servers (was: Upgrade fluorine to trusty/jessie)
GerritReady T148186: Build warm slave for Gerrit in Dallas
PhabricatorNot ready, but intended In progress: T137928: Deploy phabricator to phab2001.codfw.wmnet / T164810: Switch phabricator production to codfw
NOC / dbtreeNot ready, but intended T163141: dbtree: make wasat a working backend and become active-active
tendrilReady* dbmonitor2001 is ready but passive T149557: Site: 2 VM request for tendril (switch tendril from einsteinium to dbmonitor*), * because replication doesn't work well with events, it requires app changes. Service can be easily failed over but past monitoring data would be lost
releasesReady T171917: setup releases2001.codfw.wmnet https://gerrit.wikimedia.org/r/#/c/368527/

Related Objects

StatusSubtypeAssignedTask
Resolvedmmodell
ResolvedPaladox
ResolvedDzahn
ResolvedDzahn
ResolvedDzahn
Resolvedmmodell
ResolvedJoe
OpenNone
OpenNone
ResolvedDzahn
Resolvedfgiunchedi
ResolvedRobH
Resolvedfgiunchedi
ResolvedRobH
ResolvedArielGlenn
ResolvedRobH
Resolveddemon
ResolvedPapaul
Resolvedfaidon
Declinedfgiunchedi
ResolvedRobH
ResolvedDzahn
Resolvedakosiaris
Resolvedakosiaris
Declinedakosiaris
Resolvedfgiunchedi
Resolvedhashar
ResolvedRobH
Resolvedfgiunchedi
ResolvedCmjohnson
ResolvedCmjohnson
Resolvedhashar
ResolvedRobH
ResolvedRobH
ResolvedPapaul
ResolvedDzahn
DeclinedNone
Resolvedjcrespo
ResolvedDzahn
ResolvedDzahn
StalledNone
StalledNone
Resolvedmmodell
ResolvedRobH
ResolvedMoritzMuehlenhoff
ResolvedDzahn
InvalidNone
DeclinedDzahn
Resolvedmmodell
DeclinedNone
Resolvedmmodell
Resolvedmmodell
Declinedmmodell

Event Timeline

The reason I created this ticket is because, as a DBA, I have to support some of those services below the app layer, so I need to know the state on dallas- but it not only restricted to m[1-5]-hosted services. I included release-engineering team because many non-core services are developer-supporting tools.

Change 347892 had a related patch set uploaded (by Dzahn):
[operations/puppet@production] planet/varnish-misc: switch planet to active-active

https://gerrit.wikimedia.org/r/347892

Change 347892 merged by Dzahn:
[operations/puppet@production] planet/varnish-misc: switch planet to active-active

https://gerrit.wikimedia.org/r/347892

jcrespo renamed this task from Understand the preparedness of misc services for datacenter switchover to Provide cross-dc redundancy (active-active or active-passive) to all important misc services.May 4 2017, 1:46 PM
jcrespo lowered the priority of this task from High to Medium.

Removing tag due to change in scope of the ticket.

Krinkle updated the task description. (Show Details)
jcrespo updated the task description. (Show Details)

Change 368527 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] cache::misc: releases: add codfw backend, make active-active

https://gerrit.wikimedia.org/r/368527

Change 368527 merged by Dzahn:
[operations/puppet@production] cache::misc: releases: add codfw backend, make active-active

https://gerrit.wikimedia.org/r/368527

Dzahn updated the task description. (Show Details)
jcrespo updated the task description. (Show Details)

This could probably move forward once T218570 gets resolved.

I think those are different things. T218570: DB planning: include a writeable (?) misc DB cluster in codfw for WMCS, from my understanding, is a _new_ database misc cluster (writable) just for OpenStack.