Producing to Kafka is just one way to use EventGate. It's nice to have it as the default, except that building the librdkafka binary is not very lightway, nor does it always work so smoothly on all OS-es (e.g. MacOS). By making node-rdkafka an optional dependency (and guarding against it being missing), we should be able to more easily create development eventgate instances without needing to build librdkafka.
Description
Description
Details
Details
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Ottomata | T185233 Modern Event Platform | |||
| Resolved | Ottomata | T259202 Instrumentation development environment on EventGate server | |||
| Resolved | Ottomata | T266058 Make node-rdkafka an optional dependency of EventGate |
Event Timeline
Comment Actions
Change 635384 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[eventgate-wikimedia@master] Bump eventgate sha to 3b6023c to get node-rdkafka as an optional dependency
Comment Actions
Change 637026 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[node-rdkafka-factory@master] Initial commit
Comment Actions
Ok, I did some work today to factory out our async Kafka factory wrapper from eventgate into its own library. This makes it easier to make it an optional dependency of both eventgate and eventgate-wikimedia. This currently includes 4 changes, including one to make it all work in EventLogging as a devserver, for the parent task T259202.
These all need review, gonna bug @Milimetric and/or @Pchelolo for the first 3, and @Mholloway for the last EventLogging one.
- New node-rdkafka-factory repo and module: https://gerrit.wikimedia.org/r/c/node-rdkafka-factory/+/637026
- Make eventgate use node-rdkafka-factory: https://github.com/wikimedia/eventgate/pull/12
- Make eventgate-wikimedia use node-rdkafka-factory: https://gerrit.wikimedia.org/r/c/eventgate-wikimedia/+/635384
- Make EventLogging use eventgate-wikimedia without node-rdkafka: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/EventLogging/+/617770
These all need to be merged and published in order.
Comment Actions
Change 637514 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[integration/config@master] Add CI for node-rdkafka-factory
Comment Actions
Change 637514 merged by jenkins-bot:
[integration/config@master] Add CI for node-rdkafka-factory
Comment Actions
Mentioned in SAL (#wikimedia-releng) [2020-10-29T16:57:39Z] <James_F> Zuul: Add CI for node-rdkafka-factory T266058
Comment Actions
Change 635384 merged by Ottomata:
[eventgate-wikimedia@master] Bump eventgate to 1.5.0 to get node-rdkafka-factory as optional dependency
Comment Actions
Change 637748 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[eventgate-wikimedia@master] Fix require of namespaced @wikimedia/node-rdkafka-factory
Comment Actions
Change 637748 merged by Ottomata:
[eventgate-wikimedia@master] Fix require of namespaced @wikimedia/node-rdkafka-factory