Page MenuHomePhabricator

Use new eventgate chart release analytics for eventgate-analytics service.
Closed, ResolvedPublic5 Story Points

Description

We've refactored the eventgate-analytics chart to a more generic eventgate, and deployed an 'analytics' release of it.

Event Timeline

Ottomata created this task.May 10 2019, 3:56 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 10 2019, 3:56 PM

Hm, question. Currently mediawiki-config ProductionServices.php has:

'eventgate-analytics' => 'http://eventgate-analytics.discovery.wmnet:31192',

If we change LVS port to 33192, this will just change the LVS monitoring/pooling/depooling, right? The discovery part is just based on the domain name? Since both the old chart and the new are in kubernetes, I assume the discovered hostnames for eventgate-analytics are the same.

Will it be safe to first change the LVS port, and then make a mediawiki-config change to change the service port there? Will http://eventgate-analytics.discovery.wmnet:31192 continue to work as it does now if we change the LVS configs in puppet?

akosiaris added a comment.EditedMay 13 2019, 1:56 PM

Hm, question. Currently mediawiki-config ProductionServices.php has:

'eventgate-analytics' => 'http://eventgate-analytics.discovery.wmnet:31192',

If we change LVS port to 33192, this will just change the LVS monitoring/pooling/depooling, right?

It will change the LVS Port. That is the old one will no longer receive connections

The discovery part is just based on the domain name? Since both the old chart and the new are in kubernetes, I assume the discovered hostnames for eventgate-analytics are the same.

Yes.

Will it be safe to first change the LVS port, and then make a mediawiki-config change to change the service port there? Will http://eventgate-analytics.discovery.wmnet:31192 continue to work as it does now if we change the LVS configs in puppet?

Depends on the change. It will require a new transitional LVS service for this to be "safe". Anything else will cause an outage as the configuration of the two thing (mediawiki-config & LVS) will differ for some points in time.

Ok, in that case, it will be much less annoying to temporarily disable the events in mediawiki-config, purge the deployed eventgate-analytics chart in staging and production helm/k8s, and then upgrade the new eventgate release analytics chart to use the same 31192 LVS port.

I'll prep for this, and we can do it my afternoon today!

FYI @Pchelolo :)

Ottomata renamed this task from Change LVS port for eventlogging-analytics from 31192 to 33192 to Use new eventgate chart release analytics for eventgate-analytics service..May 13 2019, 2:23 PM
Ottomata updated the task description. (Show Details)

Change 509866 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Temporarily disable eventgate-analytics monolog events

https://gerrit.wikimedia.org/r/509866

Change 509866 merged by Ottomata:
[operations/mediawiki-config@master] Temporarily disable eventgate-analytics monolog events

https://gerrit.wikimedia.org/r/509866

Mentioned in SAL (#wikimedia-operations) [2019-05-13T17:09:54Z] <ottomata> disabling all eventgate-analytics monolog events for eventgate chart migration - T222962

Mentioned in SAL (#wikimedia-operations) [2019-05-13T17:11:25Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: disabling all eventgate-analytics monolog events for eventgate chart migration - T222962 (duration: 00m 50s)

Mentioned in SAL (#wikimedia-operations) [2019-05-13T17:41:34Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: retry - disabling all eventgate-analytics monolog events for eventgate chart migration - T222962 (duration: 00m 50s)

Change 509911 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/mediawiki-config@master] Re-enable eventgate-analytics monolog events

https://gerrit.wikimedia.org/r/509911

Change 509911 merged by Ottomata:
[operations/mediawiki-config@master] Re-enable eventgate-analytics monolog events

https://gerrit.wikimedia.org/r/509911

Mentioned in SAL (#wikimedia-operations) [2019-05-13T18:17:13Z] <ottomata> re-enabling all eventgate-analytics monolog events - T222962

Mentioned in SAL (#wikimedia-operations) [2019-05-13T18:19:21Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: re-enabling all eventgate-analytics monolog events - T222962 (duration: 00m 50s)

Ottomata set the point value for this task to 5.May 13 2019, 6:23 PM
Ottomata moved this task from Next Up to Done on the Analytics-Kanban board.

Mentioned in SAL (#wikimedia-operations) [2019-05-13T18:29:19Z] <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: re-sync: re-enabling all eventgate-analytics monolog events - T222962 (duration: 00m 49s)

Ottomata moved this task from Backlog to Done on the EventBus board.May 14 2019, 1:33 PM
Nuria closed this task as Resolved.May 14 2019, 7:57 PM