Page MenuHomePhabricator
Create Task
Maniphest T245203

Create production and canary releases for existent eventgate helmfile services
Closed, ResolvedPublic
Actions

Description

The eventgate chart has been refactored to support canary releases. Along the way, we stopped using the release name to differentiate the different app deployments of eventgate. Instead, we now use main_app.name. We should switch back to the convention of calling the production release of an app 'production', and also add a 'canary' release with a single replica.

  • eventgate-logging-external is not yet in use, so that can happen without any client disruption.

eventgate-analytics and eventgate-main are in use. For those we'll need to deploy the new production and canary releases alongside the 'analytics' and 'main' releases. To avoid port conflict, we'll have to pick new nodePorts for the 'production' release's Service. Once the new releases are deployed, we'll have to switch LVS to use the new ports.

eventgate-main will change ports from http 32192 and https 4292 to http 34192 and https 4492.
eventgate-analytics will change ports from http 31192 and https 4192 to http 35192 and https 4592.

Details

ProjectBranchLines +/-Subject
operations/deployment-chartsmaster+0 -16eventgate-main - remove unused 'main' release from staging helmfile
operations/puppetproduction+0 -29lvs: delete legacy eventgate monitors
operations/deployment-chartsmaster+0 -58Remove unused 'main' and 'analytics' releases from eventgate helmfiles
operations/puppetproduction+0 -152Remove unsed eventgate*-to-delete LVS service declarations
operations/puppetproduction+4 -4Set eventgate-*-to-delete LVS services to state: service_setup
operations/puppetproduction+4 -4Set eventgate-*-to-delete LVS services to state: lvs_setup
operations/puppetproduction+3 -3Switch change-prop and restbase event_service_uri to new TLS eventgate-main
operations/puppetproduction+8 -4Remove monitoring and alerting for eventgate to-delete services
operations/mediawiki-configmaster+0 -25Use new LVS port for eventgate-main in all wikis
operations/mediawiki-configmaster+5 -0Use new LVS port for eventgate-main in group1 wikis
operations/mediawiki-configmaster+21 -1Use new LVS port for eventgate-main in group0 wikis
operations/mediawiki-configmaster+1 -1Use new LVS port for eventgate-analytics
operations/puppetproduction+2 -2[wdqs] use https and 4592 for eventgate-analytics endpoint
analytics/refinerymaster+2 -2Change default swift event_service_url to new eventgate-analytics port
operations/homer/publicmaster+3 -1Add ports and codfw LVS IP to term eventgate-analytics in analytics-in4
operations/puppetproduction+191 -13Add new LVS services for new eventgate-main and eventgate-analytics ports
operations/deployment-chartsmaster+177 -30Configure production and canary release for eventgate-main
operations/deployment-chartsmaster+124 -21Configure production and canary release for eventgate-analytics
operations/deployment-chartsmaster+0 -48eventgate-logging-external - Remove now unused 'logging-external' release
operations/deployment-chartsmaster+165 -21Configure production and canary release for eventgate-logging-external
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Ottomata updated the task description. (Show Details)Feb 18 2020, 8:05 PM
Ottomata updated the task description. (Show Details)Feb 18 2020, 8:10 PM
gerritbot added a comment.Feb 18 2020, 8:11 PM

Change 572960 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add new LVS services for new eventgate-main and eventgate-analytics ports

https://gerrit.wikimedia.org/r/572960

gerritbot added a project: Patch-For-Review.Feb 18 2020, 8:11 PM
Ottomata triaged this task as High priority.Feb 18 2020, 8:11 PM
Ottomata updated the task description. (Show Details)
Ottomata updated the task description. (Show Details)Feb 18 2020, 8:17 PM
gerritbot added a comment.Feb 19 2020, 2:26 PM

Change 572960 merged by Alexandros Kosiaris:
[operations/puppet@production] Add new LVS services for new eventgate-main and eventgate-analytics ports

https://gerrit.wikimedia.org/r/572960

akosiaris updated the task description. (Show Details)Feb 19 2020, 2:46 PM
Maintenance_bot removed a project: Patch-For-Review.Feb 19 2020, 3:11 PM
gerritbot added a comment.Feb 19 2020, 3:21 PM

Change 573307 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Use new LVS port for eventgate-analytics

https://gerrit.wikimedia.org/r/573307

gerritbot added a project: Patch-For-Review.Feb 19 2020, 3:21 PM
gerritbot added a comment.Feb 19 2020, 3:29 PM

Change 573309 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[analytics/refinery@master] Change default swift event_service_url to new eventgate-analytics port

https://gerrit.wikimedia.org/r/573309

Ottomata updated the task description. (Show Details)Feb 19 2020, 3:35 PM
Ottomata updated the task description. (Show Details)Feb 19 2020, 3:46 PM
gerritbot added a comment.Feb 19 2020, 3:51 PM

Change 562842 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/homer/public@master] Add ports and codfw LVS IP to term eventgate-analytics in analytics-in4

https://gerrit.wikimedia.org/r/562842

gerritbot added a comment.Feb 19 2020, 3:56 PM

Change 562842 merged by Elukey:
[operations/homer/public@master] Add ports and codfw LVS IP to term eventgate-analytics in analytics-in4

https://gerrit.wikimedia.org/r/562842

gerritbot added a comment.Feb 19 2020, 3:56 PM

Change 573317 had a related patch set uploaded (by DCausse; owner: DCausse):
[operations/puppet@production] [wdqs] use https and 4592 for eventgate-analytics endpoint

https://gerrit.wikimedia.org/r/573317

Stashbot added a comment.Feb 19 2020, 4:05 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-19T16:05:20Z] <elukey> Update analytics-in4 filter term eventgate for T245203 on cr1/cr2 eqiad

Ottomata updated the task description. (Show Details)Feb 19 2020, 4:06 PM
gerritbot added a comment.Feb 19 2020, 4:07 PM

Change 573309 merged by Ottomata:
[analytics/refinery@master] Change default swift event_service_url to new eventgate-analytics port

https://gerrit.wikimedia.org/r/573309

gerritbot added a comment.Feb 19 2020, 4:08 PM

Change 573317 merged by Ottomata:
[operations/puppet@production] [wdqs] use https and 4592 for eventgate-analytics endpoint

https://gerrit.wikimedia.org/r/573317

Stashbot added a comment.Feb 19 2020, 4:24 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-19T16:24:55Z] <otto@deploy1001> Started deploy [analytics/refinery@e23918a]: Updating eventgate-analytics port (T245203) and also eventlogging whitelist

Stashbot added a comment.Feb 19 2020, 4:37 PM

Mentioned in SAL (#wikimedia-operations) [2020-02-19T16:37:22Z] <otto@deploy1001> Finished deploy [analytics/refinery@e23918a]: Updating eventgate-analytics port (T245203) and also eventlogging whitelist (duration: 12m 27s)

Ottomata moved this task from Next Up to In Progress on the Analytics-Kanban board.Feb 19 2020, 7:58 PM
Ottomata updated the task description. (Show Details)Feb 19 2020, 8:35 PM
EBernhardson added a subscriber: EBernhardson.Feb 20 2020, 12:05 AM

Re-deployed our glent esbulk oozie job against refinery versioned 2020-02-19T16.58.16+00.00--scap_sync_2020-02-19_0001. Additionally shipped an update to our airflow scheduler that changes the eventgate port used there as well.

Ottomata added a comment.Feb 20 2020, 2:34 AM

Thank you!

Ottomata updated the task description. (Show Details)Feb 20 2020, 2:35 PM
Ottomata updated the task description. (Show Details)
Ottomata added a project: Event-Platform Value Stream.Mar 2 2020, 2:56 PM
Ottomata moved this task from Backlog to In Progress Before Value Streams Kickoff (August 15th) on the Event-Platform Value Stream board.
gerritbot added a comment.Mar 2 2020, 3:08 PM

Change 573307 merged by Ottomata:
[operations/mediawiki-config@master] Use new LVS port for eventgate-analytics

https://gerrit.wikimedia.org/r/573307

Maintenance_bot removed a project: Patch-For-Review.Mar 2 2020, 3:10 PM
Stashbot added a comment.Mar 2 2020, 3:20 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T15:20:11Z] <ottomata> Use new LVS port for EventBus+monolog for eventgate-analytics - T245203

Stashbot added a comment.Mar 2 2020, 3:20 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T15:20:57Z] <otto@deploy1001> Synchronized wmf-config/ProductionServices.php: Use new LVS port for EventBus+monolog for eventgate-analytics - T245203 (duration: 00m 56s)

gerritbot added a comment.Mar 2 2020, 3:35 PM

Change 576066 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Use new LVS port for eventgate-main

https://gerrit.wikimedia.org/r/576066

gerritbot added a project: Patch-For-Review.Mar 2 2020, 3:35 PM
Ottomata updated the task description. (Show Details)Mar 2 2020, 4:18 PM
gerritbot added a comment.Mar 2 2020, 6:33 PM

Change 576066 merged by Ottomata:
[operations/mediawiki-config@master] Use new LVS port for eventgate-main in group0 wikis

https://gerrit.wikimedia.org/r/576066

Stashbot added a comment.Mar 2 2020, 6:38 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T18:38:03Z] <ottomata> using new eventgate-main LVS ports for eventbus on group0 wikis - T245203

Stashbot added a comment.Mar 2 2020, 6:39 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T18:39:13Z] <otto@deploy1001> Synchronized wmf-config/LabsServices.php: Use new LVS port for EventBus for eventgate-main on group0 wikis - T245203 (duration: 00m 58s)

Stashbot added a comment.Mar 2 2020, 6:41 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T18:41:33Z] <otto@deploy1001> Synchronized wmf-config/ProductionServices.php: Use new LVS port for EventBus for eventgate-main on group0 wikis - T245203 (duration: 00m 57s)

Stashbot added a comment.Mar 2 2020, 6:43 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T18:43:15Z] <otto@deploy1001> Synchronized wmf-config/CommonSettings.php: Use new LVS port for EventBus for eventgate-main on group0 wikis - T245203 (duration: 00m 56s)

Stashbot added a comment.Mar 2 2020, 6:45 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T18:45:12Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Use new LVS port for EventBus for eventgate-main on group0 wikis - T245203 (duration: 00m 57s)

gerritbot added a comment.Mar 2 2020, 6:48 PM

Change 576115 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Use new LVS port for eventgate-main in group1 wikis

https://gerrit.wikimedia.org/r/576115

gerritbot added a comment.Mar 2 2020, 6:51 PM

Change 576115 merged by Ottomata:
[operations/mediawiki-config@master] Use new LVS port for eventgate-main in group1 wikis

https://gerrit.wikimedia.org/r/576115

Stashbot added a comment.Mar 2 2020, 6:53 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T18:53:11Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Use new LVS port for EventBus for eventgate-main on group1 wikis - T245203 (duration: 00m 57s)

gerritbot added a comment.Mar 2 2020, 6:55 PM

Change 576117 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Use new LVS port for eventgate-main in all wikis

https://gerrit.wikimedia.org/r/576117

gerritbot added a comment.Mar 2 2020, 6:56 PM

Change 576117 merged by Ottomata:
[operations/mediawiki-config@master] Use new LVS port for eventgate-main in all wikis

https://gerrit.wikimedia.org/r/576117

Stashbot added a comment.Mar 2 2020, 6:58 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T18:58:55Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Use new LVS port for EventBus for eventgate-main on all wikis - T245203 (duration: 00m 56s)

Stashbot added a comment.Mar 2 2020, 7:01 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T19:01:55Z] <otto@deploy1001> Synchronized wmf-config/CommonSettings.php: Use new LVS port for EventBus for eventgate-main on all wikis - T245203 (duration: 00m 57s)

Stashbot added a comment.Mar 2 2020, 7:03 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T19:03:06Z] <otto@deploy1001> Synchronized wmf-config/LabsServices.php: Use new LVS port for EventBus for eventgate-main on all wikis - T245203 (duration: 00m 56s)

Stashbot added a comment.Mar 2 2020, 7:04 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-02T19:04:26Z] <otto@deploy1001> Synchronized wmf-config/ProductionServices.php: Use new LVS port for EventBus for eventgate-main on all wikis - T245203 (duration: 00m 56s)

Ottomata updated the task description. (Show Details)Mar 2 2020, 7:05 PM
Maintenance_bot removed a project: Patch-For-Review.Mar 2 2020, 7:11 PM
gerritbot added a comment.Mar 3 2020, 3:00 PM

Change 576361 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove monitoring and alerting for eventgate to-delete services

https://gerrit.wikimedia.org/r/576361

gerritbot added a project: Patch-For-Review.Mar 3 2020, 3:00 PM
Ottomata updated the task description. (Show Details)Mar 3 2020, 3:02 PM
gerritbot added a comment.Mar 3 2020, 3:08 PM

Change 576361 merged by Ottomata:
[operations/puppet@production] Remove monitoring and alerting for eventgate to-delete services

https://gerrit.wikimedia.org/r/576361

Maintenance_bot removed a project: Patch-For-Review.Mar 3 2020, 3:10 PM
Ottomata updated the task description. (Show Details)Mar 3 2020, 3:12 PM
gerritbot added a comment.Mar 3 2020, 3:25 PM

Change 576370 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Switch change-prop and restbase event_service_uri to new TLS eventgate-main

https://gerrit.wikimedia.org/r/576370

gerritbot added a project: Patch-For-Review.Mar 3 2020, 3:25 PM
gerritbot added a comment.Mar 3 2020, 4:12 PM

Change 576370 merged by Ottomata:
[operations/puppet@production] Switch change-prop and restbase event_service_uri to new TLS eventgate-main

https://gerrit.wikimedia.org/r/576370

Maintenance_bot removed a project: Patch-For-Review.Mar 3 2020, 5:10 PM
Ottomata updated the task description. (Show Details)Mar 3 2020, 5:19 PM
gerritbot added a comment.Mar 3 2020, 5:28 PM

Change 576396 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Set eventgate-*-to-delete LVS services to state: service_setup

https://gerrit.wikimedia.org/r/576396

gerritbot added a project: Patch-For-Review.Mar 3 2020, 5:28 PM
gerritbot added a comment.Mar 3 2020, 5:38 PM

Change 576396 merged by Ottomata:
[operations/puppet@production] Set eventgate-*-to-delete LVS services to state: lvs_setup

https://gerrit.wikimedia.org/r/576396

gerritbot added a comment.Mar 3 2020, 5:44 PM

Change 576402 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Set eventgate-*-to-delete LVS services to state: service_setup

https://gerrit.wikimedia.org/r/576402

gerritbot added a comment.Mar 4 2020, 2:05 PM

Change 576402 merged by Alexandros Kosiaris:
[operations/puppet@production] Set eventgate-*-to-delete LVS services to state: service_setup

https://gerrit.wikimedia.org/r/576402

Maintenance_bot removed a project: Patch-For-Review.Mar 4 2020, 2:10 PM
Stashbot added a comment.Mar 4 2020, 2:18 PM

Mentioned in SAL (#wikimedia-operations) [2020-03-04T14:18:52Z] <akosiaris> cleanup old LVS eventgate services. T245203

akosiaris updated the task description. (Show Details)Mar 4 2020, 2:19 PM

I 've merged https://gerrit.wikimedia.org/r/576402 and manually cleaned the various IPVS services that were now unknown to pybal.

Ottomata added a comment.Mar 4 2020, 2:52 PM

YES THANK YOU!

gerritbot added a comment.Mar 4 2020, 3:53 PM

Change 576865 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove unsed eventgate*-to-delete LVS service declarations

https://gerrit.wikimedia.org/r/576865

gerritbot added a project: Patch-For-Review.Mar 4 2020, 3:53 PM
gerritbot added a comment.Mar 4 2020, 3:57 PM

Change 576865 merged by Ottomata:
[operations/puppet@production] Remove unsed eventgate*-to-delete LVS service declarations

https://gerrit.wikimedia.org/r/576865

gerritbot added a comment.Mar 4 2020, 4:07 PM

Change 576871 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] Remove unused 'main' and 'analytics' releases from eventgate helmfiles

https://gerrit.wikimedia.org/r/576871

gerritbot added a comment.Mar 4 2020, 4:11 PM

Change 576871 merged by Ottomata:
[operations/deployment-charts@master] Remove unused 'main' and 'analytics' releases from eventgate helmfiles

https://gerrit.wikimedia.org/r/576871

Ottomata updated the task description. (Show Details)Mar 4 2020, 4:12 PM
Ottomata mentioned this in rDEPLOYCHARTSd7cffe327164: Remove unused 'main' and 'analytics' releases from eventgate helmfiles.Mar 4 2020, 4:13 PM
Ottomata updated the task description. (Show Details)Mar 4 2020, 4:16 PM
Ottomata moved this task from In Progress to Done on the Analytics-Kanban board.Mar 4 2020, 4:20 PM
Ottomata set Final Story Points to 21.
Ottomata updated the task description. (Show Details)Mar 4 2020, 4:55 PM
Maintenance_bot removed a project: Patch-For-Review.Mar 4 2020, 5:10 PM
Ottomata updated the task description. (Show Details)Mar 5 2020, 5:46 PM
Nuria closed this task as Resolved.Mar 5 2020, 9:14 PM
gerritbot added a comment.Mar 9 2020, 5:28 PM

Change 578365 had a related patch set uploaded (by Filippo Giunchedi; owner: Filippo Giunchedi):
[operations/puppet@production] lvs: delete legacy eventgate monitors

https://gerrit.wikimedia.org/r/578365

gerritbot added a project: Patch-For-Review.Mar 9 2020, 5:28 PM
gerritbot added a comment.Mar 9 2020, 5:31 PM

Change 578365 merged by Filippo Giunchedi:
[operations/puppet@production] lvs: delete legacy eventgate monitors

https://gerrit.wikimedia.org/r/578365

gerritbot added a comment.Mar 10 2020, 6:47 PM

Change 578581 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/deployment-charts@master] eventgate-main - remove unused 'main' release from staging helmfile

https://gerrit.wikimedia.org/r/578581

gerritbot added a comment.Mar 10 2020, 6:51 PM

Change 578581 merged by Ottomata:
[operations/deployment-charts@master] eventgate-main - remove unused 'main' release from staging helmfile

https://gerrit.wikimedia.org/r/578581

Ottomata mentioned this in rDEPLOYCHARTS067c2205cb1d: eventgate-main - remove unused 'main' release from staging helmfile.Mar 10 2020, 6:51 PM
Maintenance_bot removed a project: Patch-For-Review.Mar 10 2020, 7:11 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL