Context
In our new architecture we decided to move from using tilerator as a map tile pregeneration queuing system backed by Redis to native tegola functionality backed by Kafka.
One of the suggested ways to publish messages to the kafka deployment on wikimedia infra is EventGate, an HTTP API where event producers can POST messages to a predefined stream based on a predefined schema.
Here is the documentation around setting up instrumentation for events: https://wikitech.wikimedia.org/wiki/Event_Platform/Instrumentation_How_To
For more information about the Event Platform here is the project documentation: https://wikitech.wikimedia.org/wiki/Event_Platform
Action items
- Create schema for map tile invalidation events
- Add configuration to register the event stream