Page MenuHomePhabricator

Create EventLogging service
Closed, ResolvedPublic

Description

Having EventLogging dev server running locally is very useful for any development work that involves instrumentation. There is documentation here about one Docker flavored setup. For local-charts, it sounds like we'd need to:

  1. create a Docker image to wrap the python project
  2. add a eventlogging chart
  3. provide documentation or a default setting that sets $wgEventLoggingBaseUri = 'http://mediawiki-eventlogging:{PORT}/event.gif';

Event Timeline

kostajh created this task.May 17 2019, 5:12 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 17 2019, 5:12 PM

I started looking at this after creating a basic image for running eventlogging (https://hub.docker.com/r/kostajh/eventlogging), but got quickly overwhelmed in trying to grok the contents of helm/ (I was looking at helm/restbase config as a model). I guess this is one of the major drawbacks to using minikube for local dev; IMO compared to a docker-compose stack, it's quite cumbersome to add on your own services or modify existing ones.

Change 572212 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[releng/dev-images@master] (wip) Create image for EventLogging service

https://gerrit.wikimedia.org/r/572212

brennen moved this task from Backlog to Watching on the User-brennen board.
brennen moved this task from Backlog to In-progress on the local-charts board.

Hi interesting! FYI we are actively working on replacing the eventlogging backend with Event Platform components, namely EventGate. eventlogging-devserver might not be useful in a few quarters.

@Ottomata thanks for the heads up. Is EventGate far enough along that we should not move ahead with this image?

Hm, no I think you should go forward, although, I'm not totally sure what running eventlogging-devserver will get you inside of a container. IIRC it just prints out received events to stdout.

We build images for eventgate at https://tools.wmflabs.org/dockerregistry/wikimedia/eventgate-wikimedia/tags/. These are very usable, although you'd need to configure them a bit and also have Kafka running. However, eventgate itself can use pluggable outputs, including just to stdout, but we'd probably need to build a custom image for that, or adjust the eventgate-wikimedia one to allow for configuration to not use Kafka.

Anyway, we use eventgate a lot already for backend production things, and are just now in the process of replacing the EventLogging client side use cases of it. We hope to soon (next quarter) start producing real events to it from the EventLogging extension. See also T241241: Produce an instrumentation event stream using new EPC and EventGate from client side browsers and T238138: Vertical: Virtualpageview datastream on MEP .

kostajh moved this task from Backlog to In Progress on the MediaWiki-Docker board.Mar 29 2020, 7:08 PM
kostajh added a comment.EditedMar 31 2020, 2:08 PM

Printing to stdout is all I really need; I prefer looking at that output (and piping to jq) instead of using the browser console for checking if events are flowing as intended.

Great! Proceed! (I also prefer it your way :) )

Change 572212 merged by Brennen Bearnes:
[releng/dev-images@master] Create image for EventLogging service

https://gerrit.wikimedia.org/r/572212

Mentioned in SAL (#wikimedia-releng) [2020-03-31T15:36:37Z] <brennen> Updating dev-images docker-pkg files on contint1001 for T223643

kostajh closed this task as Resolved.Apr 1 2020, 12:09 PM
kostajh claimed this task.
brennen moved this task from Watching to Done / Defunct on the User-brennen board.Apr 3 2020, 10:25 PM