Page MenuHomePhabricator

Add monolog adapters for Eventbus
Closed, ResolvedPublic

Description

It has been decided in the parent ticket to create EventBus adapters for Monolog interfaces in order to be able to send Monolog events to Kafka via EventBus/EventGate. This ticket is created to track that work.

The plan is to create 'EventBusMonologHandler' and 'EventBusMonolog Formatter' classes and use them instead of the current 'KafkaHandler' and 'AvroFormatter' classes.

The only roadblock is that EventGate requires .$schema property with a schema URI and a .meta.stream property with a stream corresponding to the Kafka topic the messages should go to, but Monolog does not require those. I believe adding the schema URI and the stream should be a part of the configuration for the formatter. So, the formatter config would look somewhat like this:

'eventbus' => [
    'class' => 'EventBusMonologFormatter',
    'args' => [
        'ApiAction' => [
            'schema' => '/mediawiki/api/request/0.0.1',
            'stream'  => 'mediawiki-api-request'
        ],
        ...
    ]
]

The downside of having the schema and stream configured in MW config is that we will have this configuration in multiple places. We could eliminate the steam config by adopting a policy that for monolog events the stream name equals to the monolog channel name, however for the $schema config I do not see a viable solution. The schema version corresponds to a producer code, so it should be configured either within the code or within WMF-config

Thoughts?

Event Timeline

Pchelolo triaged this task as Normal priority.Feb 14 2019, 5:59 PM
Pchelolo created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 14 2019, 5:59 PM

Change 490663 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[mediawiki/extensions/EventBus@master] Create EventBus Monolog adapters.

https://gerrit.wikimedia.org/r/490663

Change 490668 had a related patch set uploaded (by Ppchelko; owner: Ppchelko):
[operations/mediawiki-config@master] [WIP]: Switch kafka logging to EventBus logging.

https://gerrit.wikimedia.org/r/490668

Ottomata raised the priority of this task from Normal to High.Feb 21 2019, 5:53 PM

Change 490663 merged by Ottomata:
[mediawiki/extensions/EventBus@master] Create EventBus Monolog adapter.

https://gerrit.wikimedia.org/r/490663

Change 490668 merged by jenkins-bot:
[operations/mediawiki-config@master] Add eventbus analytics logging alongside with kafka logging.

https://gerrit.wikimedia.org/r/490668

Change 494305 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[mediawiki/extensions/EventBus@master] Send monolog $record['context'] as event, not entire $record

https://gerrit.wikimedia.org/r/494305

Change 494305 merged by Ottomata:
[mediawiki/extensions/EventBus@master] Send monolog $record['context'] as event, not entire $record

https://gerrit.wikimedia.org/r/494305

Change 494368 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[mediawiki/extensions/EventBus@wmf/1.33.0-wmf.19] Send monolog $record['context'] as event, not entire $record

https://gerrit.wikimedia.org/r/494368

Ottomata moved this task from Backlog to Done on the EventBus board.Mar 4 2019, 10:29 PM

Change 494368 abandoned by Ottomata:
Send monolog $record['context'] as event, not entire $record

Reason:
This will go out with wmf.20 train.

https://gerrit.wikimedia.org/r/494368

Ottomata closed this task as Resolved.Mar 12 2019, 2:39 PM