We've refactored the eventgate-analytics chart to a more generic eventgate, and deployed an 'analytics' release of it.
Description
Description
Details
Details
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T185233 Modern Event Platform | |||
| Resolved | Ottomata | T201068 Modern Event Platform: Stream Intake Service | |||
| Resolved | Ottomata | T211248 Modern Event Platform: Stream Intake Service: Migrate eventlogging-service-eventbus events to eventgate-main | |||
| Resolved | Ottomata | T218346 Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main | |||
| Resolved | Ottomata | T222962 Use new eventgate chart release analytics for eventgate-analytics service. |
Event Timeline
Comment Actions
Hm, question. Currently mediawiki-config ProductionServices.php has:
'eventgate-analytics' => 'http://eventgate-analytics.discovery.wmnet:31192',
If we change LVS port to 33192, this will just change the LVS monitoring/pooling/depooling, right? The discovery part is just based on the domain name? Since both the old chart and the new are in kubernetes, I assume the discovered hostnames for eventgate-analytics are the same.
Will it be safe to first change the LVS port, and then make a mediawiki-config change to change the service port there? Will http://eventgate-analytics.discovery.wmnet:31192 continue to work as it does now if we change the LVS configs in puppet?
Comment Actions
It will change the LVS Port. That is the old one will no longer receive connections
The discovery part is just based on the domain name? Since both the old chart and the new are in kubernetes, I assume the discovered hostnames for eventgate-analytics are the same.
Yes.
Will it be safe to first change the LVS port, and then make a mediawiki-config change to change the service port there? Will http://eventgate-analytics.discovery.wmnet:31192 continue to work as it does now if we change the LVS configs in puppet?
Depends on the change. It will require a new transitional LVS service for this to be "safe". Anything else will cause an outage as the configuration of the two thing (mediawiki-config & LVS) will differ for some points in time.
Comment Actions
Ok, in that case, it will be much less annoying to temporarily disable the events in mediawiki-config, purge the deployed eventgate-analytics chart in staging and production helm/k8s, and then upgrade the new eventgate release analytics chart to use the same 31192 LVS port.
I'll prep for this, and we can do it my afternoon today!
FYI @Pchelolo :)
Comment Actions
Change 509866 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Temporarily disable eventgate-analytics monolog events
Comment Actions
Change 509866 merged by Ottomata:
[operations/mediawiki-config@master] Temporarily disable eventgate-analytics monolog events
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-05-13T17:09:54Z] <ottomata> disabling all eventgate-analytics monolog events for eventgate chart migration - T222962
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-05-13T17:11:25Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: disabling all eventgate-analytics monolog events for eventgate chart migration - T222962 (duration: 00m 50s)
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-05-13T17:41:34Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: retry - disabling all eventgate-analytics monolog events for eventgate chart migration - T222962 (duration: 00m 50s)
Comment Actions
Change 509911 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Re-enable eventgate-analytics monolog events
Comment Actions
Change 509911 merged by Ottomata:
[operations/mediawiki-config@master] Re-enable eventgate-analytics monolog events
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-05-13T18:17:13Z] <ottomata> re-enabling all eventgate-analytics monolog events - T222962
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-05-13T18:19:21Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: re-enabling all eventgate-analytics monolog events - T222962 (duration: 00m 50s)
Comment Actions
Mentioned in SAL (#wikimedia-operations) [2019-05-13T18:29:19Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: re-sync: re-enabling all eventgate-analytics monolog events - T222962 (duration: 00m 49s)