Page MenuHomePhabricator
Create Task
Maniphest T232122

Decomission eventlogging-service-eventbus and clean up related configs and code
Closed, ResolvedPublic13 Estimated Story Points
Actions

Description

Woohoo! All events are now on eventgate! https://grafana.wikimedia.org/d/000000201/eventbus?refresh=1m&panelId=1&fullscreen&orgId=1&from=now-1h&to=now

Details

ProjectBranchLines +/-Subject
operations/puppetproduction+0 -12Remove absent job camus eventbus
operations/puppetproduction+6 -1Don't render camus::job properties file if not ensure => present
operations/puppetproduction+17 -4Rename camus job eventbus to mediawiki_events
operations/puppetproduction+0 -14Remove absent job eventlogging_eventbus_job_queue
operations/puppetproduction+4 -11Remove eventbus-admins group users, use cp admins on kafka_main
operations/puppetproduction+25 -13Rename some analytics related eventbus jobs
operations/puppetproduction+1 -1Fix proper cluster assignment for kafka_main
operations/puppetproduction+3 -3Rename 'eventbus' grafana alerts and notes_urls
operations/puppetproduction+9 -9Rename eventbus cluster to kafka_main
operations/puppetproduction+0 -161Remove unused eventbus puppetization
operations/puppetproduction+0 -12calico: remove eventbus fw rules
operations/puppetproduction+2 -1profile::eventbus - Declare kafka $config in scope
operations/puppetproduction+8 -16Ensure eventlogging-service-eventbus is absent
operations/puppetproduction+1 -0Remove eventbus graphite threshold alert
operations/puppetproduction+0 -23Remove eventbus LVS hiera entries
operations/puppetproduction+0 -6Remove profile::lvs::realserver from profile::eventbus
operations/dnsmaster+4 -4Remove eventbus.svc DNS records
operations/puppetproduction+1 -30Remove LVS/pybal config for eventbus service
operations/puppetproduction+1 -3eventbus: class service::configuration has no ensure parameter
operations/puppetproduction+6 -73Remove LVS, discovery, and secondary monitoring of eventbus service
operations/puppetproduction+0 -4Remove eventbus.discovery
operations/dnsmaster+0 -2Remove eventbus.discovery info
operations/puppetproduction+1 -2Remove ensure parameter in service::configuration
operations/puppetproduction+60 -33Add $ensure params with defaults for eventlogging service - no op
mediawiki/extensions/EventBusmaster+17 -56Remove LegacyEventFactory.
operations/mediawiki-configmaster+0 -7Remove references to eventlogging-service.
operations/puppetproduction+9 -11Decomission eventlogging-service-eventbus in beta / deployment-prep
operations/mediawiki-configmaster+0 -1Remove eventbus from LabsServices.php
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Ottomata created this task.Sep 5 2019, 4:32 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 5 2019, 4:32 PM
gerritbot added a comment.Sep 5 2019, 4:33 PM

Change 534637 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/mediawiki-config@master] Remove references to eventlogging-service.

https://gerrit.wikimedia.org/r/534637

gerritbot added a project: Patch-For-Review.Sep 5 2019, 4:33 PM
Ottomata moved this task from Backlog to Next Up on the Event-Platform board.Sep 5 2019, 5:03 PM
gerritbot added a comment.Sep 5 2019, 5:05 PM

Change 534644 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/extensions/EventBus@master] Remove LegacyEventFactory.

https://gerrit.wikimedia.org/r/534644

Ottomata renamed this task from Decomission eventlogging-service-eventbus to Decomission eventlogging-service-eventbus and clean up related configs and code.Sep 5 2019, 5:06 PM
• fdans triaged this task as High priority.Sep 9 2019, 3:55 PM
• fdans added a project: Analytics-Kanban.
• fdans moved this task from Incoming to Operational Excellence on the Analytics board.
Ottomata added a subtask: T232144: Migrate all event-schemas schemas to current.yaml and materialize with jsonschema-tools and remove old schemas.Sep 9 2019, 6:46 PM
gerritbot added a comment.Sep 9 2019, 6:54 PM

Change 535265 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Remove eventbus from LabsServices.php

https://gerrit.wikimedia.org/r/535265

gerritbot added a comment.Sep 9 2019, 7:09 PM

Change 535265 abandoned by Ottomata:
Remove eventbus from LabsServices.php

Reason:
Ohh well:

15:04:56 1) WmfConfigServicesTest::testCrossRealmCompatibility
15:04:56 service keys for labs/eqiad
15:04:56 Failed asserting that two strings are identical.
15:04:56 --- Expected
15:04:56 Actual
15:04:56 @@ @@
15:04:56 etcd
15:04:56 -eventbus

https://gerrit.wikimedia.org/r/535265

gerritbot added a comment.Sep 9 2019, 7:13 PM

Change 535269 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Decomission eventlogging-service-eventbus in beta / deployment-prep

https://gerrit.wikimedia.org/r/535269

gerritbot added a comment.Sep 9 2019, 7:19 PM

Change 535269 merged by Ottomata:
[operations/puppet@production] Decomission eventlogging-service-eventbus in beta / deployment-prep

https://gerrit.wikimedia.org/r/535269

Ottomata added a comment.Sep 9 2019, 7:32 PM

eventlogging-service-eventubs decommed in beta!

akosiaris awarded a token.Sep 10 2019, 7:57 AM
gerritbot added a comment.Sep 10 2019, 5:52 PM

Change 534637 merged by jenkins-bot:
[operations/mediawiki-config@master] Remove references to eventlogging-service.

https://gerrit.wikimedia.org/r/534637

Stashbot added a comment.Sep 10 2019, 5:55 PM

Mentioned in SAL (#wikimedia-operations) [2019-09-10T17:55:08Z] <jforrester@deploy1001> Synchronized wmf-config/CommonSettings.php: T232122 Remove use of eventlogging-service (duration: 01m 03s)

Stashbot added a comment.Sep 10 2019, 5:56 PM

Mentioned in SAL (#wikimedia-operations) [2019-09-10T17:56:45Z] <jforrester@deploy1001> Synchronized wmf-config/ProductionServices.php: T232122 Stop setting production value for eventlogging-service (duration: 01m 00s)

gerritbot added a comment.Sep 10 2019, 6:22 PM

Change 535664 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add $ensure params with defaults for eventlogging service - no op

https://gerrit.wikimedia.org/r/535664

gerritbot added a comment.Sep 10 2019, 6:40 PM

Change 535664 merged by Ottomata:
[operations/puppet@production] Add $ensure params with defaults for eventlogging service - no op

https://gerrit.wikimedia.org/r/535664

gerritbot added a comment.Sep 10 2019, 6:48 PM

Change 535669 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove LVS, discovery, and secondary monitoring of eventbus service

https://gerrit.wikimedia.org/r/535669

gerritbot added a comment.Sep 11 2019, 10:13 AM

Change 534644 merged by jenkins-bot:
[mediawiki/extensions/EventBus@master] Remove LegacyEventFactory.

https://gerrit.wikimedia.org/r/534644

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.23; 2019-09-17).Sep 11 2019, 11:00 AM
gerritbot added a comment.Sep 11 2019, 1:36 PM

Change 535848 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove ensure parameter in service::configuration

https://gerrit.wikimedia.org/r/535848

gerritbot added a comment.Sep 11 2019, 1:39 PM

Change 535848 merged by Ottomata:
[operations/puppet@production] Remove ensure parameter in service::configuration

https://gerrit.wikimedia.org/r/535848

gerritbot added a comment.Sep 11 2019, 1:54 PM

Change 535852 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/dns@master] Remove eventbus.discovery info

https://gerrit.wikimedia.org/r/535852

gerritbot added a comment.Sep 11 2019, 1:55 PM

Change 535855 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove eventbus.discovery

https://gerrit.wikimedia.org/r/535855

gerritbot added a comment.Sep 11 2019, 2:39 PM

Change 535865 had a related patch set uploaded (by Jbond; owner: John Bond):
[operations/puppet@production] eventbus: class service::configuration has no ensure parameter

https://gerrit.wikimedia.org/r/535865

Stashbot added a comment.Sep 11 2019, 3:03 PM

Mentioned in SAL (#wikimedia-operations) [2019-09-11T15:03:01Z] <bblack> downtimed dns-discovery confd health checks for eventbus - T232122

gerritbot added a comment.Sep 11 2019, 3:04 PM

Change 535852 merged by BBlack:
[operations/dns@master] Remove eventbus.discovery info

https://gerrit.wikimedia.org/r/535852

gerritbot added a comment.Sep 11 2019, 3:08 PM

Change 535855 merged by BBlack:
[operations/puppet@production] Remove eventbus.discovery

https://gerrit.wikimedia.org/r/535855

gerritbot added a comment.Sep 11 2019, 3:09 PM

Change 535865 merged by Ottomata:
[operations/puppet@production] eventbus: class service::configuration has no ensure parameter

https://gerrit.wikimedia.org/r/535865

gerritbot added a comment.Sep 11 2019, 3:11 PM

Change 535669 abandoned by Ottomata:
Remove LVS, discovery, and secondary monitoring of eventbus service

https://gerrit.wikimedia.org/r/535669

gerritbot added a comment.Sep 11 2019, 3:15 PM

Change 535872 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove LVS/pybal config for eventbus service

https://gerrit.wikimedia.org/r/535872

gerritbot added a comment.Sep 11 2019, 3:45 PM

Change 535872 merged by BBlack:
[operations/puppet@production] Remove LVS/pybal config for eventbus service

https://gerrit.wikimedia.org/r/535872

gerritbot added a comment.Sep 11 2019, 3:59 PM

Change 535882 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove profile::lvs::realserver from profile::eventbus

https://gerrit.wikimedia.org/r/535882

gerritbot added a comment.Sep 11 2019, 3:59 PM

Change 535883 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/dns@master] Remove eventbus.svc DNS records

https://gerrit.wikimedia.org/r/535883

gerritbot added a comment.Sep 11 2019, 4:02 PM

Change 535882 merged by Ottomata:
[operations/puppet@production] Remove profile::lvs::realserver from profile::eventbus

https://gerrit.wikimedia.org/r/535882

gerritbot added a comment.Sep 11 2019, 4:03 PM

Change 535883 merged by BBlack:
[operations/dns@master] Remove eventbus.svc DNS records

https://gerrit.wikimedia.org/r/535883

gerritbot added a comment.Sep 11 2019, 4:08 PM

Change 535884 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove eventbus LVS hiera entries

https://gerrit.wikimedia.org/r/535884

gerritbot added a comment.Sep 11 2019, 4:18 PM

Change 535884 merged by Ottomata:
[operations/puppet@production] Remove eventbus LVS hiera entries

https://gerrit.wikimedia.org/r/535884

gerritbot added a comment.Sep 11 2019, 4:40 PM

Change 535889 had a related patch set uploaded (by BBlack; owner: BBlack):
[operations/puppet@production] calico: remove eventbus fw rules

https://gerrit.wikimedia.org/r/535889

gerritbot added a comment.Sep 11 2019, 7:52 PM

Change 535920 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove eventbus graphite threshold alert

https://gerrit.wikimedia.org/r/535920

gerritbot added a comment.Sep 11 2019, 7:52 PM

Change 535921 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Ensure eventlogging-service-eventbus is absent

https://gerrit.wikimedia.org/r/535921

gerritbot added a comment.Sep 11 2019, 7:55 PM

Change 535920 merged by Ottomata:
[operations/puppet@production] Remove eventbus graphite threshold alert

https://gerrit.wikimedia.org/r/535920

gerritbot added a comment.Sep 11 2019, 7:59 PM

Change 535921 merged by Ottomata:
[operations/puppet@production] Ensure eventlogging-service-eventbus is absent

https://gerrit.wikimedia.org/r/535921

gerritbot added a comment.Sep 11 2019, 8:01 PM

Change 535922 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] profile::eventbus - Declare kafka $config in scope

https://gerrit.wikimedia.org/r/535922

gerritbot added a comment.Sep 11 2019, 8:02 PM

Change 535922 merged by Ottomata:
[operations/puppet@production] profile::eventbus - Declare kafka $config in scope

https://gerrit.wikimedia.org/r/535922

Stashbot added a comment.Sep 11 2019, 8:21 PM

Mentioned in SAL (#wikimedia-operations) [2019-09-11T20:21:18Z] <ottomata> stopped and removed eventlogging-service-eventbus - T232122

Ottomata added a comment.Sep 11 2019, 8:31 PM

Woo hoo, eventlogging-service-eventbus is no more! Still have some various cleanup tasks to do, mostly about renaming things in puppet repo.

• Pchelolo closed subtask T232144: Migrate all event-schemas schemas to current.yaml and materialize with jsonschema-tools and remove old schemas as Resolved.Sep 11 2019, 10:11 PM
gerritbot added a comment.Sep 12 2019, 9:25 AM

Change 535889 merged by Alexandros Kosiaris:
[operations/puppet@production] calico: remove eventbus fw rules

https://gerrit.wikimedia.org/r/535889

Maintenance_bot removed a project: Patch-For-Review.Sep 12 2019, 10:10 AM
fgiunchedi mentioned this in T205870: Fully migrate producers off statsd.Sep 12 2019, 12:38 PM
Ottomata moved this task from Operational Excellence to Event Platform on the Analytics board.Sep 12 2019, 4:59 PM
Ottomata moved this task from Next Up to Ready to Deploy on the Analytics-Kanban board.Sep 12 2019, 5:37 PM
Nuria moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Sep 17 2019, 4:02 PM
gerritbot added a comment.Sep 18 2019, 1:51 PM

Change 537645 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove admin group eventbus-admins

https://gerrit.wikimedia.org/r/537645

Ottomata moved this task from Next Up to In Progress on the Event-Platform board.Sep 18 2019, 1:51 PM
gerritbot added a project: Patch-For-Review.Sep 18 2019, 1:51 PM
gerritbot added a comment.Sep 18 2019, 2:04 PM

Change 537648 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Rename eventbus cluster to kafka_main

https://gerrit.wikimedia.org/r/537648

gerritbot added a comment.Sep 18 2019, 2:09 PM

Change 537649 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove unused eventbus puppetization

https://gerrit.wikimedia.org/r/537649

gerritbot added a comment.Sep 18 2019, 2:16 PM

Change 537649 merged by Ottomata:
[operations/puppet@production] Remove unused eventbus puppetization

https://gerrit.wikimedia.org/r/537649

Ottomata added a comment.Sep 18 2019, 2:21 PM

Just ran

apt-get purge python-zmq python-tornado python-ua-parser python-urllib3 python-jsonschema python-mysqldb python-graypy python-pygments python-sprockets-mixins-statsd python-snappy python-six python-service-checker python-sqlalchemy python-statsd

on kafka-main* nodes.

gerritbot added a comment.Sep 18 2019, 2:29 PM

Change 537648 merged by Ottomata:
[operations/puppet@production] Rename eventbus cluster to kafka_main

https://gerrit.wikimedia.org/r/537648

gerritbot added a comment.Sep 18 2019, 2:51 PM

Change 537657 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Rename 'eventbus' grafana alerts and notes_urls

https://gerrit.wikimedia.org/r/537657

gerritbot added a comment.Sep 18 2019, 3:09 PM

Change 537657 merged by Ottomata:
[operations/puppet@production] Rename 'eventbus' grafana alerts and notes_urls

https://gerrit.wikimedia.org/r/537657

gerritbot added a comment.Sep 18 2019, 3:21 PM

Change 537664 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Rename some analytics related eventbus jobs

https://gerrit.wikimedia.org/r/537664

gerritbot added a comment.Sep 18 2019, 3:28 PM

Change 537669 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Fix proper cluster assignment for kafka_main

https://gerrit.wikimedia.org/r/537669

gerritbot added a comment.Sep 18 2019, 3:29 PM

Change 537669 merged by Ottomata:
[operations/puppet@production] Fix proper cluster assignment for kafka_main

https://gerrit.wikimedia.org/r/537669

gerritbot added a comment.Sep 18 2019, 3:31 PM

Change 537664 merged by Ottomata:
[operations/puppet@production] Rename some analytics related eventbus jobs

https://gerrit.wikimedia.org/r/537664

gerritbot added a comment.Sep 18 2019, 4:00 PM

Change 537645 merged by Ottomata:
[operations/puppet@production] Remove eventbus-admins group users, use cp admins on kafka_main

https://gerrit.wikimedia.org/r/537645

Ottomata moved this task from Done to Ready to Deploy on the Analytics-Kanban board.Sep 18 2019, 4:02 PM
gerritbot added a comment.Sep 18 2019, 4:09 PM

Change 537679 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove absent job eventlogging_eventbus_job_queue

https://gerrit.wikimedia.org/r/537679

gerritbot added a comment.Sep 18 2019, 4:12 PM

Change 537679 merged by Ottomata:
[operations/puppet@production] Remove absent job eventlogging_eventbus_job_queue

https://gerrit.wikimedia.org/r/537679

gerritbot added a comment.Sep 18 2019, 4:15 PM

Change 537681 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Rename camus job eventbus to mediawiki_events

https://gerrit.wikimedia.org/r/537681

gerritbot added a comment.Sep 18 2019, 4:18 PM

Change 537681 merged by Ottomata:
[operations/puppet@production] Rename camus job eventbus to mediawiki_events

https://gerrit.wikimedia.org/r/537681

gerritbot added a comment.Sep 18 2019, 4:26 PM

Change 537687 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Don't render camus::job properties file if not ensure => present

https://gerrit.wikimedia.org/r/537687

gerritbot added a comment.Sep 18 2019, 4:30 PM

Change 537687 merged by Ottomata:
[operations/puppet@production] Don't render camus::job properties file if not ensure => present

https://gerrit.wikimedia.org/r/537687

gerritbot added a comment.Sep 18 2019, 4:33 PM

Change 537688 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Remove absent job camus eventbus

https://gerrit.wikimedia.org/r/537688

gerritbot added a comment.Sep 18 2019, 4:36 PM

Change 537688 merged by Ottomata:
[operations/puppet@production] Remove absent job camus eventbus

https://gerrit.wikimedia.org/r/537688

Ottomata added a comment.Sep 18 2019, 4:39 PM

Phew ok after all those patches, I think we are good with puppet code cleanup!

Ottomata moved this task from Ready to Deploy to Done on the Analytics-Kanban board.Sep 18 2019, 4:39 PM
Ottomata set the point value for this task to 13.
Maintenance_bot removed a project: Patch-For-Review.Sep 18 2019, 5:11 PM
Ottomata moved this task from In Progress to Done on the Event-Platform board.Sep 18 2019, 6:54 PM
Nuria closed this task as Resolved.Sep 19 2019, 6:27 PM
Content licensed under Creative Commons Attribution-ShareAlike 3.0 (CC-BY-SA) unless otherwise noted; code licensed under GNU General Public License (GPL) or other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL